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ABSTRACT 


INTRODUCTION AND OBJECTIVES 


Dugway Proving Ground (DPG), located in west-central Utah, is a U.S. Department of Defense 
(DoD) test site for chemical and biological defensive testing; it is also home to multiple breeding 
pairs of golden eagles (Aquila chrysaetos). The presence of an eagle nest has the potential to stop 
military testing and training due to potential nest disturbance. It is vital to DPG and similar DoD 
installations to fully understand the status of in-use eagle nests on military lands. Using Unmanned 
Aircraft Systems (UAS) and Small Unmanned Aircraft Systems (sUAS), it may be possible to 
obtain accurate status updates of eagle nests on DoD lands more efficiently than can be 
accomplished on foot. The DPG Team designed a blind study conducting weekly golden eagle 
nest surveys on DPG to compare the effectiveness of sUAS, UAS, and ground observers. 


TECHNOLOGY DESCRIPTION 


The DPG Team conducted weekly golden eagle nest surveys on DPG during the 2019 and 2020 nesting 
seasons, using a blind study design, to compare the effectiveness of three “survey systems”: 1) a ground- 
based human observer, 2) a commercially available small Unmanned Aircraft System (sUAS; DJI 
Matrice M600 and E Mergent RC E900), and 3) a military-grade UAS platform (MO-1C Gray Eagle). 


PERFORMANCE AND COST ASSESSMENT 


No single platform was clearly superior across all seasons and periods of observation. During the 
2019 season, the sUAS observation team proved the most efficient at locating in-use and unknown 
nests, but it was more expensive than the ground observer (annual operation cost of approx. $130k 
for the sUAS, versus $20k for the ground observer). The UAS team did not observe frequently 
enough to be directly compared but appeared to be most efficient observing well-known nests with 
lots of metadata to reference. UAS costs were not directly assessed, as they operate as part of a 
larger installation program. Results differed significantly during the 2020 season after the sUAS 
platform used in 2019 was prohibited from use under the FY19 National Defense Authorization 
Act (NDAA). In 2020, the sUAS team had trouble locating nests or identifying status due to the 
change in platform/payload and was not as efficient as the ground observer. The results indicate 
that the use of an sUAS is effective but may not yet be able to cost-effectively replace ground 
monitoring. For instances where the nests are too high or terrain too rugged, sUAS are especially 
effective. As the cost of the sUAS system is reduced and the policy surrounding flying sUAS in 
DoD airspace is streamlined, it may become cost-effective compared to on-the-ground monitoring. 


IMPLEMENTATION ISSUES 


A major implementation issue with the use of sUAS in DoD airspace involves obtaining an 
Airworthiness Release (AWR) and Exemption to Policy (ETP). Both approvals take time to develop, 
review, and obtain approval. Good metadata and a member of the team that has experience with eagle 
behavior and good knowledge of the territory is key to obtaining useful data. The UAS observation 
method was the most uncertain and it proved to be unreliable as a method of weekly nest observations 
due to scheduling and payload restrictions. If the nest location was well known, the UAS was by far 
the fastest and most efficient observation method. However, the nature of the platform’s flight and 
approach patterns made detailed nest metadata critical to the success of the survey. 
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EXECUTIVE SUMMARY 
INTRODUCTION 


Dugway Proving Ground (DPG), located in remote west-central Utah, serves as a Major Range 
Test and Facility Base (MRTFB) test site for chemical and biological defensive testing, battlefield 
smokes, and obscurants. In addition, DPG is home to multiple breeding pairs of golden eagles, 
which are protected under the Bald and Golden Eagle Protection Act (BGEPA) and the Migratory 
Bird Treaty Act (MBTA). Due to these regulatory drivers, the presence of an eagle nest has the 
potential to restrict military testing and training if military activity causes nest disturbance. It is 
therefore vital to the function of DPG and similar DoD testing and training ranges to fully 
understand the location and status of in-use eagle nests or territories on military lands. 


Current survey techniques of nesting golden eagles on DPG and active DoD training properties 
require long-distance viewing of each nest from the ground at least weekly, which is both time- 
consuming, costly, and may not always provide a definitive status of nest changes (Weissensteiner 
and Poelstra 2015) due to limited visibility from the ground. Using both military-grade and 
commercial Unmanned Aircraft Systems (UAS)/Small Unmanned Aircraft Systems (sUAS), it is 
possible to obtain highly accurate locations and status updates of in-use golden eagle nests on DoD 
lands, data that is critical to maintaining open testing and training ranges for as much time as 
possible in the eagle nesting season (Chabot and Bird 2015, Hodgson et al. 2016, Junda et al. 
2015, Weissensteiner and Poelstra 2015). The SUAS/UAS can be used to detect new nests in areas 
that are difficult to see or access from the ground, as well as count nestlings and track nesting 
milestones in a less time-invasive manner not previously possible (Weissensteiner and Poelstra 
2015). All of this can be accomplished with greatly reduced disturbance to the nesting pairs 
(through reduced time in the area), crucial to the success of sensitive nesters such as the golden 
eagle (Chabot and Bird 2015, Scobie and Hugenholtz 2016, Weissensteiner and Poelstra 2015). 


OBJECTIVES 


Select Engineering Services (SES), in coordination with DPG, the Rapid Integration and 
Acceptance Center (RIAC), Threat Systems Management Office (TSMO), and HawkWatch 
International (HWT), hereinafter referred to as the DPG Team, developed this technology 
demonstration to demonstrate the use of sUAS/UAS as a cost-effective way to monitor golden 
eagles on military installations. Currently it takes approximately 960 hours to conduct on-the- 
ground monitoring of eagle nests for approximately six months (based upon actual numbers 
provided by DPG and SES experience). The primary objective was to reduce the total hours 
required to monitor the nests and evaluate the cost-effectiveness of using sUAS/UAS over in- 
person ground based monitoring. Other objectives of the demonstration included the following: 


e Efficiencies of scale to monitor more nests in less time for a lower cost 

e Demonstrated the effectiveness of sUAS/UAS versus on-the-ground observation 
e Improved quality of still photos and videos for further analysis 

e Demonstrated use of sUAS/UAS as safe and low disturbance to the eagles 


e Demonstrated use of sUAS/UAS to enable greater access to the nest and ability to collect 
more detailed data 
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e Demonstrated use of sUAS/UAS as more efficient in finding nests in particular settings 
such as hidden/remote terrain 


TECHNOLOGY DESCRIPTION 


The DPG Team conducted weekly golden eagle nest surveys on DPG during the 2019 and 2020 
nesting seasons to compare the effectiveness of three “survey systems”: 1) a ground-based human 
observer, 2) a commercially available sUAS, and 3) one or more military-grade UAS platform(s). 
These nest surveys were conducted as a blind study; none of the three teams of observers 
encountered one another in the field. None exchanged information concerning the nest surveys, 
and none had any access to data generated by surveys conducted by any of the other teams. 


sUAS/UAS Platform Descriptions 


In the 2019 study season, a DJI Matrice sUAS platform was procured with an approved DoD 
software patch, through the Army Ground Aerial Target Control System (AGATCS), in 
partnership with the TSMO. DJI Zenmuse Z30 cameras were used as the operational payload, with 
32x optical zoom allowing for long-range HD observations. This system was approved for use 
during the 2019 nesting season. However, in December 2019 the National Defense Authorization 
Act (NDAA) was signed which stated that DoD could not operate or enter into or renew a contract 
for the procurement of a covered unmanned aircraft system that is manufactured in a covered 
foreign country. Foreign country was defined as People’s Republic of China. sUAS platforms by 
DoD programs except in the case of sUAS counter-defense testing. This language was strictly 
interpreted to prevent the use of the modified DJI Matrice that was used the previous year. This 
led the DPG Team to seek out another option. Through discussions with TMSO, an alternative 
sUAS was identified for use, the E Mergent RC E900 platform. The two E900s were integrated 
with Sony FCB-EV7520A camera payloads. 


DPG is also home to the Rapid Integration and Acceptance Center (RIAC) which was established 
by the Project Manager Unmanned Aircraft Systems to create a single location where unmanned 
and manned aircraft could complete end-to-end testing and integration. DPG coordinated with 
RIAC to conduct monitoring flights when they were not conducting mission flights. RIAC 
identified a military UAS platform that would be most suitable for this type of flight plan: the MQ- 
1C Gray Eagle. 


Survey Methods 


The DPG Team conducted weekly golden eagle nest surveys on DPG during the 2019 and 2020 
nesting seasons to compare the effectiveness of three “survey systems”: 1) a ground-based human 
observer, 2) a commercially available small Unmanned Aircraft System, and 3) one or more 
military-grade UAS platform(s). The ground-observer was a single vehicle-based biologist; the 
sUAS system included three personnel (commercial pilot, observer, and biologist for video 
monitoring); and the military UAS required three personnel (military pilot, military camera 
operator, and civilian biologist for video monitoring). All three biologists associated with each 
survey approach had comparable familiarity with existing DPG eagle territories and previous eagle 
nest monitoring experience. 
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The DPG Team generated territory-specific data sheets that specify known nests within 11 known 
territories on DPG, nest survey order based on historic use patterns, and other detailed survey 
instructions. The UAS observation team conducted a separate territory and nest survey effort 
designed to test the platform’s capabilities as a precision, on demand survey system for areas with 
little to no ground access. Two senior DPG environmental staff provided weekly oversight of data 
collected by all three survey systems and collated survey results to ascertain the weekly “benchmark” 
status of each territory, and to ensure survey protocols were properly followed. Camera data from a 
subset of nests were also used to provide retrospective verification and correction of weekly status. 


Each annual survey was segmented into two time-periods: the “early season”, defined as 10 
February—14 April (9-10 weeks) and the “late season”, defined as 15 April-30 June (12 weeks). 
The primary goal of the early season surveys was to document “Occupied Territories” (pair of 
adults present), “In-Use Nests” (nest with eggs or young), and “Vacant Territories” (no in-use 
nests and no adult birds in territory), while the goal of late season surveys was to track In-Use Nest 
to “Success” (at least one nestling surviving to 80% fledge age) or “Failure” (terms adapted from 
Steenhof et al. [2017]). During the early season, all 11 territories were surveyed every two weeks 
(e.g., 6 one week, five the next). In contrast, late season surveys were restricted to In-Use Nests 
found during the early season surveys, plus any territories with unresolved status. 


For ground and sUAS survey systems, independent teams attempted to survey each nest from 
outside an 800-m horizontal protective buffer (as widely prescribed on public lands and suggested 
by Romin and Muck [2002]). Biologists associated with each survey system recorded eagle 
response behavior (i.e., any agitation or flushing observed while conducting nest surveys). 


Surveys conducted for each platform followed specific territory and nest survey protocols to ensure 
comparable methods were employed and to facilitate the capture of detailed time-keeping records 
associated with specific survey tasks. Biologists for each survey system recorded total daily 
“Survey Time”, which began upon departure from English Village (ground and sUAS) or initiation 
of take-off from the DPG airfield (UAS) and continued until return to same location at the end of 
the survey effort. For each platform, “Territory Survey Time” was noted, starting when entering 
into a 2km buffer around known nests associated with each territory, and continuing until status 
was confirmed, and the survey was complete, or a 3-hr time limit was reached. Time spent within 
each territory involved in “Area Search” (general scanning overall area for adult and nesting 
evidence), “Nest Search” (prescribed order survey of known nests), and “Extended Search” 
(searching for new or uncatalogued nests) was also recorded. Surveyors used standardized 
terminology established in this document and photographic guides to age eagle nestlings (Driscoll 
2010, Hoechlin 1976). 


PERFORMANCE ASSESSMENT 

Time to Early Season Status 

The ground observer and the sUAS took roughly the same amount of time to determine early 
season territory occupancy each year, even with the change in sUAS platform in 2020. The UAS 
platform was rarely able to make definitive determinations within the given time allowed, and its 


data was subsequently not comparable to the other platforms. Most of the time required by both the 
sUAS and ground observer was taken by drive time between nests in a territory. 
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Time To Determine Early Season Status 
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Figure ES-1. Chart Showing the Mean Time in Minutes (+SD) Each Platform Required 
to Determine the Early Season Occupancy of a Territory. 


Date of In-Use Nest Identification 


The sUAS platform was roughly equal with the ground observer at detecting in-use nests in 2019. 
The sUAS platform was able to survey almost every known nest in each assigned territory each 
week, making it more likely for the platform to detect an incubating eagle within a week of nesting 
initiation. The ground observer, on the other hand, was forced to rotate which nests were surveyed 
during each visit due to long hikes over rough terrain. The 2019 UAS was typically not able to 
identify nest activity due to the low resolution of its payload and the lack of precise nest metadata 
provided to its pilots and was not compared. In 2020 the sUAS was again roughly equal to the 
gound observer in nest detection time, despite the sUAS’s drop in payload resolution. This 
indicates that the advantage of visiting more nests outweighed the disadvantage of poor payload 
resolution. In 2020, the UAS platform was forced to cease surveys before nests became in-use, 
removing it from this comparison. 
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Figure ES-2. Chart Showing the Mean Number of Days (+SD) from Nest 
Initiation to Activity Detection by Each Platform. 


Date of Nest Failure or Fledge Identification 


In 2019, the ground observer and sUAS platforms very nearly tied for speed at detecting nest 
failure or fledge. Both had sufficient quality optics and visitation rates to detect the lack of chicks 
in a nest almost immediately. The UAS was not compared due to lack of data. By contrast, the 
ground observer was much faster to detect nest failure or fledge in 2020 than the sUAS. The 
sUAS’s loss of payload resolution made it difficult for the platform to identify the presence or 
absence of small chicks in a nest. Again, the UAS was forced to cease surveys before nests became 


in-use, removing it from this comparison. 
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ES Figure ES-3. Chart Showing the Mean Number of Days (+SD) from Nest 
Failure or Fledge to Detection by Each Platform. 


Time to Nest Status Update 


These models illustrate the differences in nest observation efficiency both between platforms and 
between years. In 2019, the sUAS displayed a clear efficiency advantage over the ground observer, 
requiring much less observation time to make a nest status determination, especially in the early 
nesting season. This is because of the sUAS’s ability to view directly into a nest bowl, eliminating 
the need to wait for a possible incubating eagle to stand or stretch in order to come into view from 
the ground. As the nesting season went on and chicks got larger, they were easier to see from 
below, decreasing the advantage of the sUAS. In 2020, the sUAS retained a small advantage over 
the ground observer, but the poor resolution of the payload meant that the platform had to observe 
each nest for much longer in order to determine if an object perceived to be in the nest was actually 
an eagle or if it was simply a pixilation artifact. In addition, the ground observer in 2020 took more 
time on average as the season went on, mostly due to the fact that one of the in-use nests was 
located in a high, partially obscured cliff cavity, preventing easy observation at any time. 
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Figure ES-4. Comparison of the Efficiency of a Ground Observer in Minutes (+ 95% 
Confidence Intervals) vs an sUAS at Determining Any Given Nest’s Status During 2019. 


The sUAS took vastly less time to determine status during early season, while the ground observer gained 


efficiency as unused nests were eliminated from the survey round. 
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Figure ES-5. Comparison of the Efficiency of a Ground Observer in Minutes (+ 95% 
Confidence Intervals) vs an sUAS at Determining Any Given Nest’s Status During 2020. 


The sUAS took less time to determine status on average, but the sUAS’s efficiency advantage was much 
less pronounced than during 2019. 


Hypothetical Mission Impact in Days 


The hypothetical mission impact of nest area closures and openings is a measure of the precision 
of each platform in detecting nest status changes. Ideally, an in-use nest will be detected, and an 
area closure enacted immediately (within one week or survey period) upon the start of incubation, 
and the area opened again immediately upon the failure or fledge of the nest. This minimizes 
potential mission impacts of unnecessary range closures as well as protecting the nest against 
potential disturbance. 


In 2019, both the ground observer and sUAS platform were slow to detect the activity of RN315, 
leaving it unprotected for several weeks, although the sUAS was one week quicker in detecting 
the nest’s activity. In addition, the ground observer designated RN315 as fledged prematurely by 
one week. This resulted in the sUAS being cumulatively more precise in designating range closures 
in 2019 than the ground observer. 
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In 2020, the data is complicated by the fact that both the ground observer and the sUAS monitored 
nests that the other platform was not capable of reaching, leaving only one nest for direct 
comparison. The ground observer was slow to detect the activity of RN248A by several weeks but 
caught the fledge of the nest immediately. The rough terrain and lack of road access in RN248A’s 
territory required the ground observer to ration which nests received a survey visit each week, 
which resulted in the in-use nest going undetected for several weeks. The sUAS detected the 
activity of RN029 immediately but was slow to confirm the failure of the nest by several weeks. 
The low resolution of the platform payload in 2020 made it much more difficult to identify the 
presence or absence of young chicks from the nest bowl than the presence or absence of an 
incubating adult. Both platforms were equally precise at determining the status of RN321. 


Hypothetical Mission Impact of Nest Protections 
Unprotected Nest Empty Nest Protected 
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Figure ES-6. Chart Showing the Hypothetical Impact of Nest Area Closures/Openings by 
Platform and Year. 


Quality of Photo/Video Documentation 


The DPG Team conducted a qualitative evaluation of how well each platform was able to document 
nest status in both 2019 and 2020. In 2019, the sUAS platform delivered high resolution video and 
photo documentation of all nests viewed, as well as footage of the surrounding cliffs for the team to 
search for previously unknown nests. The photo/video documentation allowed the data managers to 
confirm nest status before putting range closures in place. By contrast, the UAS platform in 2019 
was equipped with a low-resolution payload that hampered the observer’s ability to identify nests 
with confidence. The UAS’s thermal imaging capability did allow the observer to spot chicks in in- 
use nests, however. In 2020, the imaging capabilities of the sUAS and UAS platforms were reversed. 
The replacement sUAS platform had a downgraded payload resolution due to downlink limitations. 
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In addition, it did not have onboard data recording capabilities; the observation team had to record 
footage from the ground station, and as a result encountered significant issues with poor reception 
and signal corruption. These limitations resulted in the sUAS team requiring more time on site to 
determine nest status, and in many cases prevented the team from calling a certain nest status at all. 
On the other hand, the UAS platform received a significant upgrade to its payload systems, allowing 
for HD image recording. Unfortunately, the UAS team was forced to cease survey operations after 
only 2 weeks of monitoring due to the COVID-19 crisis, and the observation team was unable to 
obtain enough data to present a significant image comparison. 


Figure ES-7. A Photo of RN315 As Taken by the sUAS Platform in 2019 
(DJI Matrice 600 Pro with Zenmuse Z30 payload). 


The photo shows an adult golden eagle sheltering a chick less than a week old. 


Figure ES-8. A Series of Two Photos Taken by a Ground Observer at Two Different Zoom 
Levels of RN046. 


The first photo shows the distance the observer must view the nest from, as well as the height of the nest on the 
cliff face. The second photo shows how the ground observer must view the nest from below, even at high zoom. 
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Figure ES-9. A Video Still Taken from the Footage of the UAS Observer Using Thermal 
Imaging in 2019, Showing RN315 with a Nearly 9-week-old Chick. 


The chick is shown as a pale (warm) shape in the middle of a cool cliff face. This image shows the 
challenges the UAS observer faced with low resolution imagery, and how thermal imaging was able to 
partially compensate in the late nesting season. 


Figure ES-10. A Video Still Taken from the Footage of the sUAS Observer in 2020 (E900 
with Sony FCB-EV7520A), Viewing Nest RN248. 


The photo shows the low resolution and video artifacts that inhibited the sUAS’s ability to make accurate 
nest activity identifications in 2020. 
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Additional Metrics for Success 


The results of the mixed-effect logistic regression models indicate that the ground observer and 
sUAS platform had essentially equal (high) accuracy in determining the status of any given nest 
in 2019. Both platforms were more than 85% likely to determine nest status accurately and 
confidently in the early season, growing to more than 95% accurate in the late season. This can be 
attributed to the experience and skill of the ground observer, as well as the ability of the sUAS 
platform to take high quality documentation photos and video for review. In 2020, the sUAS 
platform suffered greatly from the drop in payload resolution. It began strongly and was still 
confidently able to determine when nests were not in-use. However, it struggled to accurately 
identify when nests were in-use and to determine the difference between an incubating nest and a 
nest with chicks. The ground observer, on the other hand, displayed almost the same results as in 
2019. 
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Figure ES-11. Comparison of the Accuracy of a Ground Observer (+ 95% Confidence 
Intervals) vs an SUAS at Determining Any Given Nest’s Status During 2019. 


Both platforms gained accuracy as the season went on, remaining roughly equal in accuracy. Error 
margins represent 95% confidence intervals. 
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Figure ES-12. Comparison of the Accuracy of a Ground Observer (+ 95% Confidence 
Intervals) vs an sUAS at Determining Any Given Nest’s Status During 2020. 


The ground observer gained accuracy during the season, mirroring 2019, while the sUAS lost accuracy. 
This represents the challenges of identifying an in-use nest’s status with a low-resolution payload. Error 
margins represent 95% confidence intervals. 


When determining which platform is most efficient and cost-effective, it is important to take total 
territory coverage into account, especially in the early nesting season when looking to identify in- 
use nests. The sUAS platform was able to survey approximately twice as many nests each week 
than the ground observer in both 2019 and 2020. This grants the sUAS more confidence in 
accurately calling the occupancy status of early season territories, as well as giving it an advantage 
in locating in-use nests promptly. This played out in 2019 as the sUAS located one in-use nest a 
full week before the ground observer. The ground observer had to rotate which nests to observe 
each week due to time pressure and the lengthy hikes needed to cover some territories. By being 
unable to cover all known nests each week, the ground observer was slower to identify an in-use 
nest, leaving it potentially vulnerable to disturbance. 
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Figure ES-13. Chart Showing the Mean Number of Nests (+SD) Surveyed Each Week 
During the Early Season. 


COST ASSESSMENT 


The table below provides a side-by-side comparison of the two sUAS, UAS, on-the-ground 
monitor and helicopter. Due to the unique circumstances faced during this demonstration, it is 
assumed that the additional cost for Government support for waiver and oversight would not be 
necessary for future projects. 


Table ES-1. Cost Comparison of Methods with Government Support for Waiver 


18,139 5,500 | NA NA $ 216,000 
NA 


NA 


Binoculars, scope 2,000 | $ 2,000 
Transportation (to-from 
observation locations ) 


Monitoring Labor 55,346 
Waiver Development (contractor ) 7,000 
*Govt. waiver prep and 
sponsor/training/oversight of flights 79,557 


* Cost specific to this project as a result of DoD Memo banning COTS sUAS and FY19 NDAA prohibiting the 
procurement or operation of Chinese made sUAS. 
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The most significant cost driver is DoD Army sUAS policy. The platform/payload must meet 
Department of Army cyber security requirements. Each platform must undergo an evaluation and 
provide an Exemption to Policy (ETP) also referred to as a waiver, to operate in restricted airspace. 
The cost associated with obtaining an ETP includes labor to develop the ETP application package 
(Standard Operating Procedures, System Safety Management Plan, and Memos), upload through 
SIPR, and track through the system. The waivers are only for six months. In addition, each sUAS 
must have an Airworthiness Release. 


The prohibition on foreign made sUAS impacts costs because the U.S. market is still developing 
and has not yet produced a cost competitive commercial sUAS to compete with foreign made 
sUAS. As the U.S. market matures and purchases increase, this will begin to bring the costs down. 


Costs were not directly assessed for much of the UAS platform team, as the operation of the UAS 
platforms are part of a larger installation program and were made available for the study on a non- 
interference basis. 


Costs are based on golden eagle monitoring on a MRTBF with vast distances between nests and 
rugged terrain. Breeding season begins in February and goes through June, allowing for a range of 
temperatures and weather conditions. 


The cost comparison assumes that both the sUAS and on-the-ground monitoring start and return 
from the same location. The time to survey nests and return to the starting location has been recorded. 


Industry data on life expectancy of an sUAS is between one and two years or ~200 flights. When 
first envisioning this project, it was thought that the sUAS would be most cost-effective and could 
replace the current approach of on-the-ground monitoring. The results from the demonstration 
show that the use of an sUAS is a good tool to have available but may not yet be able to cost- 
effectively replace on-the-ground monitoring. However, the sUAS provides benefits over that of 
the on-the-ground monitor such as photos which can be used to study and accurately age the chicks. 
The sUAS can also cover more territory than an on-the-ground monitor and access nests that are too 
high or at a difficult angle to see from the ground. In addition, some of the costs associated with this 
demonstration can be reduced or eliminated in future use cases, such as costs associated with 
government waiver oversight. As the cost of the sUAS system is reduced and the policy surrounding 
flying the sUAS in DoD airspace is streamlined or eliminated, it may become cost-effective compared 
to on-the-ground monitoring. The use of sUAS compared to helicopter use is cost competitive. 


IMPLEMENTATION ISSUES 


A major implementation issue with the use of sUAS in DoD airspace involves obtaining an AWR 
and Exemption to Policy (ETP). This policy seeks to prevent the use of foreign made sUAS based 
on cybersecurity vulnerabilities. Both approvals take time to develop, review, and obtain approval. 
Under the 2019 DoD Army policy, a modified DJI Matrice 600 was used to monitor golden eagles 
after obtaining an ETP. However, because of the FY 2020 NDAA, Section 848 Prohibition on 
Operation or Procurement of Foreign-Made Unmanned Aircraft Systems, the procurement or use 
of Chinese made sUAS was prohibited. This led to the use of a different sUAS platform. Each 
change of platform requires anew AWR and ETP. The ETP can only be granted for six months at 
a time and the time to obtain an approval may take 45-60 days minimum. 
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The sUAS platform used for observation and data collection has real potential for becoming a 
quick, cost-effective way to monitor the progress of nesting birds in hard-to-reach places. For this 
platform to work efficiently and in a manner that collects the best data, some points described 
below had been found over the past two years to help improve the use of this platform. 


Good metadata and a member of the team that has experience with eagle behavior and good 
knowledge of the territory is key to obtaining useful data. A trained eye for these sUAS platforms 
was able to compensate for finding nests with sometimes less than ideal video streaming quality 
or lighting situations. A profile containing pictures from multiple angles and distances and GPS 
coordinates of known nest in the territory was critical to confirming nest locations and to reduce 
the time that the sUAS team spent locating nests during a flight. 


The UAS observation method was the most uncertain upon the start of the 2019 nesting season, 
and it proved to be unreliable as a method of weekly nest observations due to scheduling and 
payload restrictions. RIAC allowed the use of their students and platforms on an as-available basis, 
and other missions, weather concerns, or maintenance frequently bumped nest observations off the 
schedule. In addition, the sections of airspace the students were scheduled for were often not over 
the desired areas of range containing eagle nests to be surveyed. It became clear to the DPG Team 
that rather than using the UAS as a stand-alone observation method, it would be more useful as a 
precision tool for obtaining rapid nest observations in areas with no ground access or with critical 
range use planned within the next two weeks. 


If the nest location was well known, the UAS was by far the fastest and most efficient observation 
method due to lack of drive and hiking time between observation points. However, the nature of 
the platform’s flight and approach patterns made detailed nest metadata critical to the success of 
the survey. UAS platforms must determine an approach angle and loiter point for a territory 
observation from several kilometers away, requiring the observation team to have precise knowledge 
of the nest’s location reference data, elevation, aspect, bearing from the ideal viewpoint, and 
topographical inclusion before the nest can be found. The UAS observation team found that 
landscape level photos combined with detailed descriptions of a nest’s location were sufficient to 
allow for a successful survey. 
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10 INTRODUCTION 
1.1 BACKGROUND 


Dugway Proving Ground (DPG), located in remote west-central Utah, serves as a Major Range 
Test and Facility Base (MRTFB) test site for chemical and biological defensive testing, battlefield 
smokes, and obscurants. In addition, DPG is home to multiple breeding pairs of golden eagles, 
which are protected under the Bald and Golden Eagle Protection Act (BGEPA) and the Migratory 
Bird Treaty Act (MBTA). Due to these regulatory drivers, the presence of an eagle nest has the 
potential to restrict military testing and training if military activity causes nest disturbance. It is 
therefore vital to the function of DPG and similar DoD testing and training ranges to fully 
understand the location and status of in-use eagle nests or territories on military lands. 


Current survey techniques of nesting golden eagles on DPG and active DoD training properties 
require long-distance viewing of each nest from the ground at least weekly, which is both time- 
consuming, costly, and may not always provide a definitive status of nest changes from incubation 
to hatching, number of nestlings, nestling condition, etc. (Weissensteiner and Poelstra 2015) due 
to views from the ground. Finding new nests is driven by bird behavior, luck, or significant 
amounts of time and effort spent traversing rugged terrain on foot. Currently, ranges are required 
to err on the side of caution if the status of an eagle nest is uncertain, which may lead to increased 
range closure times. Using both military-grade and commercial Unmanned Aircraft Systems 
(UAS)/Small Unmanned Aircraft Systems (SUAS), it is possible to obtain highly accurate locations 
and status updates of in-use golden eagle nests on DoD lands; data that is critical to maintaining 
open testing and training ranges for as much time as possible in the eagle nesting season (Chabot 
and Bird 2015, Hodgson et al. 2016, Junda et al. 2015, Weissensteiner and Poelstra 2015). The 
sUAS/UAS can be used to detect new nests in areas that are difficult to see or access from the 
ground, as well as count nestlings and track nesting milestones in a noninvasive manner not 
previously possible (Weissensteiner and Poelstra 2015). All of this can be accomplished with 
greatly reduced disturbance to the nesting pairs (through reduced time in the area), crucial to the 
success of sensitive nesters such as the golden eagle (Chabot and Bird 2015, Scobie and 
Hugenholtz 2016, Weissensteiner and Poelstra 2015). 


1.2 OBJECTIVE OF THE DEMONSTRATION 


Select Engineering Services (SES), in coordination with DPG, the Rapid Integration and 
Acceptance Center (RIAC), Threat Systems Management Office (TSMO) and HawkWatch 
International (HWT), hereinafter referred to as the DPG Team, developed this technology 
demonstration to demonstrate the use of sUAS/UAS as a cost-effective way to monitor golden 
eagles on military installations. Currently, it takes approximately 960 hours to conduct on-the- 
ground monitoring of eagle nests for approximately six months (based upon actual numbers 
provided by DPG and SES experience). Much of this time is spent traveling to and from remote 
sites either by foot or truck. The primary objective was to reduce the total hours required to monitor 
the nests and evaluate the cost-effectiveness of using sUAS/UAS over in-person ground-based 
monitoring. Other objectives of the demonstration included the following: 


e Efficiencies of scale to monitor more nests in less time for a lower cost 


e Demonstrated the effectiveness of sUAS/UAS versus on-the-ground observation 


e Improved quality of still photos and videos for further analysis 
e Demonstrated use ofsUAS/UAS as safe and low disturbance to the eagles 


e Demonstrated use of sUAS/UAS to enable greater access to the nest and ability to collect 
more detailed data 


e Demonstrated use of sUAS/UAS as more efficient in finding nests in particular settings 
such as hidden/remote terrain 


1.3 > REGULATORY DRIVERS 


Golden eagles are protected under BGEPA and the MBTA. These protections require eagles to be 
protected against “take,” defined as “[to] pursue, shoot, shoot at, poison, wound, kill, capture, trap, 
collect, molest or disturb.” In addition, the protections against “take” extend to the nests and eggs 
of eagles, as well as the adult birds. The conservation status of the golden eagle is currently stable 
globally, but declining regionally, in both western and eastern U.S. populations (Hoffman and 
Smith 2003, Katzner et al. 2012, Smith 2008). Additionally, a recent review by the U.S. Fish and 
Wildlife Service (USFWS) upheld concern over the population status of the golden eagle (USFWS 
2016a), leading to a "no net loss" standard in the recent revision to the "Eagle Rule" (USFWS 
2016b). This standard essentially prohibits any "take" of golden eagles or their nests without 
offsetting such take through the permitting framework. To prevent accidental take of golden 
eagles, DPG is obligated to monitor its existing population of nesting golden eagles and avoid 
disturbance, except in cases of specific permitted activities. As DPG can monitor its golden eagle 
population more effectively and efficiently, the installation will be able to operate with minimal 
concerns for incurring accidental take. 


2.0 TECHNOLOGY/METHODOLOGY DESCRIPTION 
2.1 TECHNOLOGY/METHODOLOGY OVERVIEW 


The DPG Team conducted weekly golden eagle nest surveys on DPG during the 2019 and 2020 
nesting seasons to compare the effectiveness of three “survey systems”: 1) a ground-based human 
observer, 2) a commercially available SUAS, and 3) one or more military-grade UAS platform(s). 
These nest surveys were conducted as a blind study; none of the three teams of observers 
encountered one another in the field. None exchanged information concerning the nest surveys, 
and none had any access to data generated by surveys conducted by any of the other teams. 


Golden eagle nest surveys are typically performed by one observer that travels to 11 territories to 
observe and note status of nesting. A second observer is sometimes used for safety if the 
terrain/location warrant it. Equipment includes high power binoculars/scope. All observations are 
documented, on paper, but no pictures or videos are taken. On-the-ground observers are required 
to undergo unexploded ordinance training (UXO) administered by Dugway range safety officers 
as well as basic coordination, navigation, and radio ops training specific to DPG. 


The sUAS platform DJI Matrice 600 (Figure I) was originally proposed for this project. However, 
the 23 May 2018 DoD Memo, “Unmanned Aerial Vehicle Systems Cybersecurity Vulnerabilities” 
was issued, suspending the use of Commercial Off-the-Shelf (COTS) sUAS. In response, a DJI 
Matrice was procured with an approved DoD software patch, through the Army Ground Aerial Target 
Control System (AGATCS), in partnership with the Threat System Management Office (TSMO). DJI 
Zenmuse Z30 cameras were used as the operational payload, with 32x optical zoom allowing for 
long-range HD observations. This system was approved for use during the 2019 nesting season. 


Figure 1. DJI Matrice 600 Pro 


However, in December 2019 the National Defense Authorization Act (NDAA) was signed which 
called for a complete ban on the procurement and/or use of Chinese made sUAS platforms by DoD 
programs except in the case of sUAS counter-defense testing (Appendix 2). This language was 
strictly interpreted to prevent the use of the modified DJI Matrice that was used the previous year. 
This led the DPG Team to seek out another option. Through discussions with TMSO, an alternative 
sUAS was identified for use, the E Mergent RC E900 platform (Figure 2). The two E900s were 
integrated with Sony FCB-EV7520A camera payloads. 
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Figure 2. E Mergent RC E900 


Table 1 below provides a comparison of the two platforms. The two payloads were technically 
comparable but the Matrice was designed as an integrated system whereas the E900 was integrated 
with a camera payload after manufacture. The primary differences between the two were the 
transmission distances, operating frequencies, and the SD card which was in the camera on the 
Matrice and in the controller tablet of the E900. These differences led to a significant degradation 
of photo quality from the E900. 


Table 1. A Comparison of the Two sUAS Platforms: 
The Modified DJI Matrice 600 in 2019 and the E900 Used in 2020 


DJI Matrice 600 E900 
Max Transmission Distance 3.1 miles 1.5 miles 
Frequency 5.8 GHz 2.4 GHz 
Payload Specifications 30x Optical Zoom + 6x Digital zoom| 30x Optical zoom + 12x Digital zoom 
2.13 Mp 2.13 Mp 
1080p 480p (functional resolution) 
Tablet Tap Zoom supported Separate controller for camera 
Camera and Interface Abilities Auto/Manual — Focus, Shutter Auto/Manual - Focus, Shutter 
Priority, Bright Compensation priority, Bright Compensation 
SD card Supported SD card in controller tablet 
Batteries 6x 5700 mAh 1x 6S 2200 mAh battery 
Battery weight — 2490 g (+/-20g) 
Estimated Flight Time (minutes) ~25 min ~20 min 
Max Takeoff Weight 15.5 kg 10.5 kg 
(payload capacity) Payload Capacity - 5.5 kg Payload Capacity — 4.5 Kg 
Max Wind Resistance 18 mph 20 mph 


The sUAS team consisted of a pilot-in-command, operator, and observer who was also an 
ornithologist with in-depth knowledge of the Dugway range and eagles in the area. All three 
members of the team completed training before the study began consisting of UAS flight theory, 
platform components, flight controls, operation, troubleshooting techniques, and supervised flight 
training administered by TSMO. All members of the team underwent unexploded ordinance 
training (UXO) administered by Dugway range safety officers as well as basic coordination, 
navigation, and radio ops training specific to DPG. 


DPG is also home to the Rapid Integration and Acceptance Center which was established by the 
Project Manager Unmanned Aircraft Systems to create a single location where unmanned and manned 
aircraft could complete end-to-end testing and integration. DPG coordinated with RIAC to conduct 
monitoring flights when they were not conducting mission flights. RIAC identified a military UAS 
platform that would be most suitable for this type of flight plan. There are approximately 24 
Continental United States Army installations with UAS that could potentially be used for monitoring 
of raptors or other natural resources as part of their ongoing training of UAS pilots. 


The UAS was operated by UAS pilots, supported by an ornithologist who provided coordinates 
and identification/analysis of nests. 


2.2 TECHNOLOGY/METHODOLOGY DEVELOPMENT 


The DPG Team’s technical approach was to demonstrate the cost-effective use of sUAS/UAS to 
monitor golden eagles by a direct blind comparison between on-the-ground monitoring and 
sUAS/UAS overflights. A flow diagram for the methodology proposed can be found in Appendix 
3, and a detailed overview of the methodology developed can be reviewed in section 5.4 (Field 
Testing). 


2.3 ADVANTAGES AND LIMITATIONS OF THE TECHNOLOGY/ METHODOLOGY 


Currently, on-the-ground monitoring includes one observer visiting each of the 11 territories once or 
more per week until occupancy is established, the in-use nest is identified, or sufficient monitoring and 
time of year eliminates the possibility of initiation of nesting activity. In-use nests are tracked with in- 
person visits once per week for the duration of the breeding season until nest fledge or demonstrable 
failure. This method is relatively reliable with repeated visits, but is time-consuming, costly, and may 
not always provide a definitive status on a given day of observation. Also, the changes in nest status 
are not immediately available, which can cost the military valuable operation days on closed ranges. 


The demonstration studied two different sUAS platform/payload systems: a DJI Matrice 600 Pro 
with a Zenmuse Z30 payload in the 2019 nesting season, and an E Mergent RC E900 prototype 
with a Sony FCB-EV7520A payload in the 2020 nesting season. The two platforms performed 
very differently during each nesting season, and each showed its own strengths and limitations. 


The DJI Matrice is a commercial off-the-shelf product, and as such was well designed and easy to 
use. The Matrice was stable and reliable in the field, with robust trouble-shooting procedures and 
documentation. The integrated Zenmuse camera system provided HD on-board photo and video 
recording of nests over distances of over 800 meters. The Matrice maintained flight control and 
video transmission over distances of up to 3 miles, far greater than typical line of sight limits, 
allowing for remote or obscured nests to be observed efficiently. However, the bulky battery 
system and low tolerance for temperature extremes limited the Matrice in particularly rugged 
conditions. The Matrice is also manufactured in China, and as such it is currently restricted from 
use in DoD restricted airspace except for use in counter-sUAS defense applications. Therefore, 
this system is no longer available for use in DoD restricted airspace. 


The E Mergent RC E900 is a prototype sUAS platform developed for custom DoD applications in 
partnership with TSMO. The main advantage of the E900 over the commercial DJI Matrice is that it 
is American made, and as such is currently eligible for consideration of the DoD issued flight waiver. 
It is also highly customizable to a specific project’s needs and can be modified and repaired at will 
with sufficient technological expertise. However, the E900 is not a fully realized commercial 
product, and as such it was less refined and reliable than the Matrice. The E900 also lacked the ease 
of use of the DJI payload controls and software. It was determined that the current iteration of the 
E900 used for the demonstration lacked the long-range transmission capabilities required, as well as 
possessing inadequate video stabilization and control operations. Integrating a third-party payload 
into the E900 system resulted in limited video resolution and recording capabilities. The after- 
manufacture integration of the camera onto the platform created some photo quality issues. The SD 
card was located in the controller tablet instead of on the camera, therefore photos/videos would be 
transmitted to the SD card, which required the images to be reduced from 1080p to 480p. 


Furthermore, both sUAS platforms have risk minimization features. Both platforms include user 
friendly, intuitive flight controls with programmable altitude restrictions, horizontal geofencing, 
and GPS hold mode to insure platform stability in windy weather. An added GPS failsafe includes 
a return to home/auto land switch on the controller that initiates an autopilot to land the platform 
at its original takeoff location with no input from the pilot. Both platforms include real time 
information such as distance from controller, heading indicator, altitude above takeoff location, 
and battery performance all on one display. This heads-up information allows separation from 
nesting areas to help prevent “take” of the birds, or to stay clear of restricted airspace. 


The MQ-1C Gray Eagle was also used when available for nest monitoring in both the 2019 and 
2020 nesting seasons (Figure 3). The Gray Eagle fleet on DPG underwent a significant upgrade to 
their payload systems during the fall of 2019, resulting in much higher resolution video for the 
2020 season. This is the first time a military UAS platform has been used to monitor nesting 
raptors. It was theorized that the high speed and very long observation ranges of the platform 
would allow it to survey many territories both quickly and thoroughly, with no impact to ground 
operations in the area. The thermal imaging capabilities of the Gray Eagle platform were also 
intriguing as a method of detecting chicks (before they develop insulating adult plumage). It was 
also theorized that student pilots training with RIAC would benefit from the high difficulty of 
locating eagle nests in cryptic terrain, especially given a mission structure mimicking what they 
would encounter in theater. Because the Gray Eagle was not always available for monitoring, it 
was difficult to obtain comparable data. 


Figure 3. MQ-1C Gray Eagle 


Although not specifically included in the demonstration, helicopters can also be used to monitor 
nests. This method, as with the other surveying techniques, has its pros and cons. The greatest 
advantage of using a helicopter as an observation platform is its ability to move quickly from territory 
to territory. With a trained spotter on board and with the use of optics, nest status can be quickly 
recorded from an advantageous perspective in hard to reach places. A trade-off of the helicopters 
speed and mobility is the noise of the aircraft increasing the potential for nest and eagle disturbance. 


These disturbances may skew the data as disruptions of a nesting pair of eagles may result in failure 
of a nest in a worst-case scenario, especially when monitoring frequently throughout a season as 
required by DPG’s ongoing range activities. The higher perspective and fast movement of the 
helicopter may also limit the observer’s ability to search the area for eagles when first entering a 
territory, which makes determining early season occupancy difficult. This practice of using optics 
to search an area when first entering a territory has proven effective in the past of helping an 
observer’s ability to track an eagle to its nest. Furthermore, from a logistical standpoint, the 
scheduling availability of range airspace for the helicopter to be cleared to fly would be 
inconsistent at best, as the nesting period of the eagles fall during a typically busy time for range 
operations. As a final consideration, there is a significant risk to human health and safety when 
monitoring eagle nests via a helicopter, due to the inherent dangers of flying close to cliff faces 
and low through rugged terrain. The very high cost of both pilot and helicopter needs to be taken 
into consideration when contemplating using a helicopter for territory observations and data 
collection. 


3.0 PERFORMANCE OBJECTIVES 


The following performance objectives were developed as the primary criteria by which the DPG 
Team could assess the success of the sUAS/UAS platforms as nest monitoring tools. The specific 
benchmarks identified are focused on reduction of time required to monitor nests, increase in 
number of nests surveyed, and increase in precision of results, all in comparison to the standard 


ground observer. 


Table 2. 


Performance Objectives 


Performance Objective 


Metric 


Data Requirements 


Success Criteria 


Total amount of time 
necessary to determine the 
early season status of all 
known nests 


Total number of in-use 
nests identified 


In-use nest date of 
identification 


Nest failure date of 
identification 


The total amount of time 
necessary to update the 
status of in-use nests 


The number of previously 
unknown alternate eagle 
nests found within the 11 
territories 


Differences in hypothetical 
mission impact (in days) 
and nest protection based 


lifted due to monitoring 


Quality of video/photos to 
document occupancy and 
status 


on restrictions imposed and | failure/fledge 


Quantitative Performance Objectives 


Amount of time necessary 
for the method of observation 
to determine early season 
status of all known nests 


Number of in-use nests the 
method of observation is able 
to identify 


Accuracy of the method of 
observation in determining 
nest activity 


Accuracy of the method of 


observation in detecting nest 
failure 


Amount of time necessary 
for the method of observation 
to update the status of a 
known in-use nest 


Ability of the method of 
observation to detect new or 
previously unknown eagle 
nests 

Precision of the method of 


observation in determining 
dates of activity and 


Qualitative Performance Objectives 


Ability of the method of 
observation to gather proof 
of nest activity status 


Exact start and ending 
times of daily survey for 
each method of 
observation 


Weekly report of nest 
activity status for each 
known territory by each 
method of observation 


Dates of territory visits 
recorded in conjunction 
with nest status for each 
method of observation 


Dates of territory visits 
recorded in conjunction 
with nest status for each 
method of observation 


Exact start and ending 
times for territory and nest 
visits for each method of 
observation 


Each new nest observed 
recorded by each method 
of observation 


e Dates of perceived 
activity and failure/ 
fledge recorded by each 
method of observation 
True dates of nest 
activity and failure/ 
fledge recorded by 


camera equipment 
directly observing nest 


Photo/video 
documentation of each in- 
use nest by SUAS/UAS 
and in-person observers 


sUAS/UAS time to 
determine early season 
status < or = in-person 
time to determine status 
Total number of nests 
identified by sUAS/UAS > 
or = number of nests 
identified in-person 


Date of identification the 
same or earlier for 
sUAS/UAS than in-person 


Date of identification the 
same or earlier for 
sUAS/UAS than in-person 


sUAS/UAS total time 
spent to update nest status 
< or = time spent in-person 


Total number of new nests 

found by sUAS/UAS > or 

= number of nests found 

in-person 

e sUAS/UAS indicated 
days of mission impact 
< or = in-person 
indicated days of 
mission impact 

e sUAS/UAS indicated 
days of nest protection 
most similar to days of 
actual nest activity 


sUAS/UAS gather higher 
quality video and photos to 
document nest activity than 
in-person observers 


4.0 SITE DESCRIPTION 
4.1 SITE LOCATION AND HISTORY 


Dugway Proving Ground is located on the southern end of the Great Salt Lake Desert and covers 
798,214 acres, serving as a test site for chemical and biological defensive testing, battlefield 
smokes and obscurants (Figure 4). DPG was activated for military use in 1942 to serve as a proving 
ground for the testing of chemical weapons. Chemical, biological, and explosives testing have 
been conducted within DPG. Open air testing of chemical and biological agents was conducted 
until 1969, after which all chemical and biological warfare testing was confined to sealed 
chambers. Open-air testing of decontamination methods, contamination avoidance, and evaluation 
of threat dissemination methods, including the use of biological and chemical simulants in place 
of biological and chemical agents, has been done at DPG since 1969. In addition, DPG was first 
opened to Army Reserve Component training in 1969. Ground training on DPG includes a wide 
variety of activities at DPG, ranging from 12-soldier weekend activities to 3,000- to 4,000-soldier 
tactical exercises lasting up to several weeks. 
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Figure 3.1.1 
Dugway Proving Ground Location 
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Figure 4. Map of DPG Showing Location Within UT 
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4.2 SITE CHARACTERISTICS 


DPG is within the Great Basin subdivision of the Basin and Range Physiographic Province. The 
Basin and Range Physiographic Province includes parts of Idaho, Oregon, Nevada, Utah, 
California, Arizona, New Mexico, and northern Mexico. This province is characterized by a series 
of mostly isolated north-south trending mountain ranges that are separated by wide desert plains 
(Press and Siever 1982). 


Most of DPG lies within the Great Salt Lake Desert, with mountains and low-lying basin areas 
covering remaining portions of DPG. DPG is bordered to the northeast by the Cedar Mountains 
and to the south by a series of ranges and valleys, the closest of which is the Dugway Range. The 
Onaqui Mountains and Davis Mountain lie to the east of DPG. The Deep Creek Range lies to the 
west and marks the boundary of the Great Salt Lake Desert. The Stansbury Mountains lie to the 
northeast of the Cedar Mountains. 


Relatively extensive basin areas are broken by the topographic relief of the Cedar Mountains, Little 
Davis Mountain, Simpson Buttes, Camels Back Ridge, Wig Mountain, Granite Peak, and Sapphire 
Mountain. There are no large perennial surface water bodies that lie within or border DPG. 
However, two large playas are located in the western and southern portions of DPG, the DPG 
Playa, and the Downwind Grid Playa. Vegetated and non-vegetated sand dunes are also located in 
the eastern and central portions of DPG and along DPG’s northern and western boundaries. 


DPG is in a semi-arid, continental, steppe region, or high desert known as the Great Basin Desert. 
This region is often referred to as a cold desert due to its mid-latitude location. Typically, winters 
are cold; summers are hot and dry with a high evaporation rate; and most precipitation falls in the 
spring (Dugway Proving Ground 2003a). 


Other weather characteristics typical of the DPG area include frequent electrical storms and 
occasional dust storms in summer and temperature inversion conditions in winter. Temperature 
inversion conditions occur when cold Arctic air spills into the area, wind speed is low, and contrary 
to the normal pattern, air temperature increases with height above the ground surface. Surface 
airflow is reduced and any tendency toward reduced air quality is aggravated under these 
conditions (Dugway Proving Ground 2003a). 


Weather patterns at DPG are influenced by terrain. Most of DPG is relatively flat because it 
consists of a former lakebed (the former Lake Bonneville of which the Great Salt Lake is asmall 
remnant). Interspersed in the flat terrain are abrupt, often pinnacle-like mountains, which are 
cooler and receive more precipitation than surrounding flatlands. They influence local weather 
patterns by channeling winds and promoting up- and down-slope conditions in mornings and 
evenings, respectively (Dugway Proving Ground 2003a). 


Local wind patterns are governed by differential heating and cooling of higher elevations relative 
to flatlands and by regional weather. These patterns usually include the onset of southeasterly or 
southerly downslope flow at night that persists into morning, which transitions into northwesterly 
through northerly flow with daytime heating. There are two periods of relative atmospheric 
stability in early morning and early evening hours. These patterns are marked in summertime but 
weak or absent in winter, due to differences in the amount of heat in the form of solar radiation 
received seasonally and the tendency of snow to reflect solar radiation away during winter 
(Dugway Proving Ground 2003a). 
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Monthly average temperatures for the Ditto Technical Center for the period 1950 to 1998 range 
from 77.9° F in July, which is the hottest month, to 27° F in January, which is the coolest. Daily 
extremes for each month show a substantial range. For example, for July the daily extreme high is 
109° F and the extreme low is 37° F. Similarly, the daily extreme range for January is 91° F. Large 
temperature fluctuations recorded between day and night and seasonally are typical of the area’s 
arid continental climate (Dugway Proving Ground 2003a). 


Precipitation data for the Ditto Technical Center for the period 1950 to 1998 show that mean annual 
precipitation is about 8 inches with a low of about 3 inches and a high of about 15 inches. Wettest 
months are March, April, and May, followed by October. Snowfall occurs November through 
March; however, snow may persist at mountain elevations for much longer periods than on 
flatlands (Dugway Proving Ground 2003a). 


The average warmest month is July, and January is the average coldest month. The highest 
recorded temperature was 109° F in 1998, and the lowest recorded temperature was -29° F in 1989. 
May is the average wettest month. 


Unusual or severe weather conditions, such as fog and cloud ceilings, which limit visibility, occur 
most often during winter. Thunderstorms or electrical storms occur during summer (Dugway 
Proving Ground 2003a). 


The map below (Figure 5) shows the 2018 known eagle nest territories. These territories are 
significant distances between them and cover a wide range of terrain and are limited by road access. 


Dugway Proving Ground e 
2018 Eagle Nest Territories 
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Figure 5. Map of Golden Eagle Nests and Territories at DPG 
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5.0 TEST DESIGN 
5.1 CONCEPTUAL TEST DESIGN 


The DPG Team conducted weekly golden eagle nest surveys on DPG during the 2019 and 2020 
nesting seasons to compare the effectiveness of three “survey systems”: 1) a ground-based human 
observer, 2) a commercially available small sUAS, and 3) one or more military-grade UAS 
platform(s). The ground-observer was a single vehicle-based biologist; the sUAS system included 
three personnel (commercial pilot, observer, and biologist for video monitoring); and the military 
UAS required three personnel (military pilot, military camera operator, and civilian biologist for 
video monitoring). All three biologists associated with each survey approach had comparable 
familiarity with existing DPG eagle territories and previous eagle nest monitoring experience. 


The DPG Team generated territory-specific data sheets that specify known nests within 11 known 
territories on DPG (Figure 5), nest survey order based on historic use patterns, and other detailed 
survey instructions. The ground observer and the sUAS observation team attempted to survey the 
same 11 territories relatively close in time, but the biologists associated with each survey system 
remained blind to each other’s findings. The UAS observation team conducted a separate territory 
and nest survey effort designed to test the platform’s capabilities as a precision, on demand survey 
system for areas with little to no ground access. Two senior DPG environmental staff provided 
weekly oversight of data collected by all three survey systems and collated survey results to 
ascertain the weekly “benchmark” status of each territory, and to ensure survey protocols were 
properly followed. Camera data from a subset of nests were also used to provide retrospective 
verification and correction of weekly status. Generally, all surveys conducted by the ground and 
sUAS observation teams occurred on Monday and Tuesday of each week, with Wednesday 
available as a weather or other make-up day, and Thursday for data review by senior staff. 
Individual territory surveys by the ground observer and sUAS were staggered to avoid overlap in 
time. The military UAS observation team conducted surveys as available. Each survey round of 
co-monitored territories (i.e., territories “shared” between survey systems, which may vary based 
on stage of season and performance of individual systems) were completed by all three platforms 
within 36 hours of each other, unless weather delays or other unforeseen issues occurred for one 
or more platform. 


Each annual survey was segmented into two time-periods: the “early season”, defined as 10 
February—14 April (9-10 weeks) and the “late season”, defined as 15 April-30 June (12 weeks). 
Survey seasons were selected based on 12 February-19 March “core” egg laying dates (1.e., 90% 
of observed values, with 5% tails removed) back-dated from 1,038 Utah desert nests (K. Keller, 
unpublished data). The early season end date was set to allow at least two survey rounds after the 
latest likely lay date based on past data. Similarly, the late season end date was set to encompass 
80% fledge age based on the latest likely egg laying date. The primary goal of the early season 
surveys was to document “Occupied Territories’, “In-Use Nests”, and “Vacant Territories”, while 
the goal of late season surveys was to track In-Use Nest to “Success” (at least one nestling 
surviving to 80% fledge age) or “Failure” (terms adapted from Steenhof et al. [2017]). During the 
early season, all 11 territories were surveyed every two weeks (e.g., six one week, five the next). 
In contrast, late season surveys were restricted to In-Use Nests found during the early season 
surveys, plus any territories with unresolved status. 
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For ground and sUAS survey systems, independent teams attempted to survey each nest from 
outside an 800-meter horizontal protective buffer (as widely prescribed on public lands and 
suggested by Romin and Muck [2002]). If topography prevented clear viewing at that distance, 
these survey systems were authorized to approach to within 400 meters of a nest, but only as close 
as necessary to confirm nest status. The “incursion” view distance was recorded. Closer 
approaches to confirm inactive nests were allowed to facilitate area nest searching. Biologists 
associated with each survey system recorded eagle response behavior (i.e., any agitation or 
flushing observed while conducting nest surveys). 


Surveys conducted for each platform followed specific territory and nest survey protocols to ensure 
comparable methods were employed and to facilitate the capture of detailed time-keeping records 
associated with specific survey tasks. Biologists for each survey system recorded total daily 
“Survey Time”, which began upon departure from English Village (ground and sUAS) or initiation 
of take-off from the DPG airfield (UAS) and continued until return to same location at the end of 
the survey effort. For each platform, “Territory Survey Time” was noted, starting when entering 
into a 2-km buffer around known nests associated with each territory, and continuing until status 
was confirmed, and the survey was complete, or a 3-hr time limit was reached. Time spent within 
each territory involved an “Area Search” (general scanning overall area for adult and nesting 
evidence), “Nest Search” (prescribed order survey of known nests), and “Extended Search” 
(searching for new or uncatalogued nests) was also recorded. Surveyors used standardized 
terminology established in this document and photographic guides to age eagle nestlings (Driscoll 
2010, Hoechlin 1976). The specific protocols associated with each survey period and specific 
survey tasks are detailed in the following section. These approaches were further reinforced by 
complementary flow charts (Appendix 3), and territory-specific data sheets. 


5.2 BASELINE CHARACTERIZATION AND PREPARATION 


Three surveyors with high levels of previous experience surveying golden eagle nests for this study 
were selected. One ground observer had 7 years of golden eagle nest survey experience in the 
West Desert prior to the start of this study, including 4 years surveying nests on DPG. The sUAS 
observer had 4 years of golden eagle nest survey experience, all conducted on DPG. The UAS 
observer possessed 6 years of golden eagle nest survey experience in the West Desert, overlapping 
with 4 years surveying in other parts of Utah, and 1 year of experience in southern Idaho before 
this study. Prior to the start of each field season, each observer reviewed the data collection 
protocols and held a briefing to discuss field procedures and relevant survey details associated 
with individual territories. Datasheets included territory and nest maps to aid all observers in 
locating nests (Table 3). At the end of year one, a debriefing was held to discuss survey issues 
encountered by each observer. It was found that high angle, distant views from the UAS platforms 
and limited UAS observer experience with DPG nests seriously limited nest locations in year one. 
As a result, “mission packets” were created prior to the start of year two, which included 
recommended UAS approaches, viewing angle, nest elevation offset, and nest photos, where 
available. 
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Table 3. Status of Golden Eagle Territories on DPG 


2015-2018. Nest status was determined by ground observers using binoculars and 60x spotting scopes, 
equipped with maps showing nest locations and photos of a sub-sample of nests. Territories were 
considered to contain an “In-Use Nest” if observers detected evidence of a breeding attempt (an adult 
eagle in incubating posture on a nest, or nestlings), or occupied if a pair of eagles were present in the 


territory. 

Territory Name # of nests* | 2015 2016 2017 2018 

Six Horse pass 6 Occupied In-Use Nest Occupied Occupied 
Cane Springs 2 Occupied Occupied Occupied In-Use Nest 
North Camelback 12 In-Use Nest In-Use Nest In-Use Nest Occupied 
North Granite 8 In-Use Nest In-Use Nest In-Use Nest In-Use Nest 
Roughhaul Rd 5 In-Use Nest Vacant Vacant Occupied 
South Camelback 15 Occupied In-Use Nest Occupied In-Use Nest 
South Granite 13 Occupied Occupied Vacant Occupied 
Simpson Butte 8 Unknown** In-Use Nest In-Use Nest Unknown** 
West Vertical Grid 4 In-Use Nest In-Use Nest In-Use Nest In-Use Nest 
Wig Mountain 6 In-Use Nest In-Use Nest In-Use Nest Occupied 
Five Mile Hill 7 Vacant Vacant Occupied Occupied 


* The number ofnest location records is approximate, and includes all documented nest locations, which may include 
some that have collapsed and no longer exist, or some duplicate nest records. 


** Ground observers were not permitted to access Simpson Butte due to unexploded ordnance in the vicinity during 
2015 and 2018. 


5.3 DESIGN AND LAYOUT OF TECHNOLOGY AND METHODOLOGY 
COMPONENTS 


This project was designed to compare the cost-effectiveness and technical capability of the ground 
monitor versus the sUAS versus the UAS when monitoring nesting golden eagles. The hypothesis 
was that sUAS would provide a cost-effective means of monitoring eagles over the traditional 
ground monitoring due to the amount of area that can be covered in a short amount of time, as well 
as the use of HD cameras that are able to be positioned at various altitudes and angles to provide 
the best look into the nest. The sUAS is also a relatively inexpensive, easy to operate platform. 
Figure 6 below shows the 11 territories that were monitored each season. 


The methodology used was based on best practices in the eagle monitoring arena to ensure a blind 


study with verifiable results. All results were analyzed in conjunction with costs to determine the 
most cost-effective solution. 
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Figure 6. Known Golden Eagle Nest Locations Associated With 11 Territories on DPG 
5.4 FIELD TESTING 


The following instructions were provided to the observers to guide field observations, in addition 
to the simplified flow chart of methods provided in Appendix 3. 


Early Season: Surveyors will attempt to search all 11 territories every two weeks, with 6 and 5 
territories surveyed on alternating weeks, utilizing up to 2 survey days per week, and up to 3 hours 
search time per territory. Territories will be grouped by location and the ground observer and sUAS 
team will alternate territory groups between days. Territory survey order within each group will 
be based upon DPG identified priority areas and optimal time of day viewing conditions. Surveyors 
will follow strict survey protocols detailed below and visualized by flowcharts in Appendix 3. 


Step 1: Determine necessary survey effort level at each territory: 


e If first territory survey, or no In-Use Nest found during previous survey(s), attempt 
complete search below (Step 2—7) until territory status is determined or 3-hour time limit 
is reached for that territory. 


e If In-Use Nest present on previous visit, complete Steps 2—4, and then proceed to “In- 
Use Nest” protocol. This includes nests for which the previous status was “previously 
In-Use unknown” (see description in “In-Use Nest” protocol), until status is resolved. 
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e Ifan In-Use Nest in this territory was previously determined as “failed”, do not survey. 


Step 2: Record daily “Survey Start Time” when leaving English Village (Ground/sUAS) or upon 
take-off (UAS). 


Step 3: Record individual “Territory Survey Start Time” when within 2-km buffer around known 
nests. 


Step 4: Navigate to within 800 m+ or adjusted viewing location of first ranked nest(s) (order 
prescribed based on survey history). 


Step 5: “Area Search”: record stationary time spent searching for free-flying eagles within territory 
or on ridges (up to 15 mins). Time does not include approach drive/fly time to stationary point, but 
adults observed during approach are noted (please indicate if observed on approach). 


e No adults seen, proceed to “Nest Survey”. 


e One adult seen, proceed to “Nest Survey”, but adjust nest survey order to begin with known 
nest(s) within 250 m of adult, survey all other nests in prescribed order. 


e Two adults seen off-nest, search any known nest(s) and substrate visible within 250 m of 
adults for signs of nest tending/eggs. If both eagles remain off-nest for 15 mins, territory 
survey ends, with times noted, and territory status will be recorded as “Territory Occupied, 
no In-Use Nest”. 


Step 6: “Nest Survey”: attempt to search all known nests in pre-determined order, with visit- 
specific modifications as warranted. Each nest is viewed until status is confirmed (i.e., “In-Use” 
[adult, egg, or young on nest], “inactive” [not In-Use, based on clearly visible nest bowl/platform, 
or condition inadequate for nesting], “Unknown” [30 mins viewing viable nest, status 
unresolved], “Time expired” [territory survey end time reached before status confirmed], or “Not 
Surveyed”). Evidence of tending/greenery is noted. Record all nest survey time undifferentiated 
by nest but note if full 30 mins required for individual nests. 


e Attempt to survey all known nests in prescribed order (based on nest histories, but adjusted 
if presence of adult(s) detected during this or previous survey, or recent evidence of 
tending), until “In-Use Nest” (eggs/nestling/incubating adult) found, two adults seen off- 
nest, all nests surveyed, or 3 hr time limit reached. If all nests were not surveyed during a 
previous survey due to time expiration, resume from previous position. 


e Upto 5 mins may be spent scanning nearby cliff faces and ridgetops upon arrival at each 
new viewpoint. 


e Ifan In-Use Nest is found, proceed to “In-Use Nest” protocol (survey ends there, with end 
times noted). Territory status will be recorded as “Occupied-In-Use Nest”. 


e If two adults seen off-nest, go to protocol under “Two adults seen”, under “Step 5: Area 
Search” (survey ends there, with end times noted). 


e Ifall nests surveyed without detecting an In-Use Nest, and a single adult has been seen on 
this or any previous survey, record end of Nest Survey time and proceed to “Extended 
Search”. 
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e If all nests surveyed without detecting an In-Use Nest, and no eagles were detected in the 
territory, record territory status as “Vacant Territory”, record end of Nest Survey time and 
proceed to next territory. 


Step 7: “Extended Search”: attempt to find new nests when only one adult seen and known nests 
are not In-Use. Record search time, including time hiking if relevant, and indicate area searched 
on map. 


e Search potential nest substrate and ridges within 250 m of previous adult sightings(s) for 
potential alternate nests, until an In-Use Nest is found (go to “In-Use Nest” protocol), two 
adults are seen off-nest, the area surrounding the sighting has been adequately searched 
(may include hiking for ground observer), or time expires. Add any new nests found to the 
end of next territory visit search order unless evidence of tending. 


e Ifan In-Use Nest is found, proceed to “In-Use Nest” protocol (survey ends there, with times 
noted). 


e If two adults seen off-nest, go to protocol under “Two adults seen”, under “Step 5: Area 
Search” (survey ends there, with end times noted). 


Step 8: Record “Territory Survey End Time” when leaving 2-km buffer around each territory 
(NOTE: 3 hr territory limit begins upon entering 2-km threshold, but may expire during nest 
survey, extended search, or In-Use Nest monitoring). 


Step 9: Repeat Steps 3-8 until all territories allotted for survey in a given day are visited. Ground 
and sUAS may not survey territories allotted for other survey days to maintain separation of survey 
platforms. 


Step 10: Record daily “Survey End Time” on return to English Village or runway. 


Late Season: Surveyors will attempt to visit all In-Use Nests identified in early season every week, 
plus any carryover territories with ambiguous status until resolution is reached, utilizing up to two 
survey days per week, and up to a time limit specific to each territory (between 1 and 4 hrs). 
Territories will be grouped by location and the ground observer and sUAS team will alternate 
territory groups between days. Territory survey order within each group will be based upon DPG 
identified priority areas and optimal time of day viewing conditions. Surveyors will follow strict 
survey protocols, detailed below, and visualized by flowcharts in Appendix 3. 


Step 1: Determine necessary survey effort level at each territory: 


e If In-Use Nest present on previous visit, complete Steps 2-4, and then proceed to “In-Use 
Nest” protocol. This includes nests for which the previous status was “previously In-Use 
unknown” (see description in “In-Use Nest” protocol), until status is resolved. 


e Ifan In-Use Nest in this territory was previously determined failed, do not survey. 


e If early season surveys found no adults in the territory (i.e., “Vacant Territory”), or 
produced two or more separate sightings of two off-nest adults (i.e., “Occupied Territory, 
no In-Use Nest”), do not survey. 
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e If early season surveys found only one adult and no In-Use Nest, additional surveys may 
or may not be warranted. If all known nests were surveyed definitively on at least two 
separate visits, and at least one “Extended Search” was conducted of substrate within 250m 
of any adult sighting(s), no additional survey is necessary. Otherwise, follow Early Season 
protocol until these criteria are met or In-Use Nest is found. 


Step 2: Record daily “Survey Start Time” when leaving English Village (Ground/sUAS) or runway 
(UAS). 


Step 3: Record individual “Territory Survey Start Time” when within 2-km buffer around known 
nests. 


Step 4: Approach first nest viewing location of In-Use Nest and proceed to “In-Use Nest” protocol. 


Step 5: Record “Territory Survey End Time” when leaving 2-km buffer around each territory, or 
when time expires (NOTE: 3-hr territory limit begins upon entering 2km threshold but may expire 
during survey). 


Step 6: Repeat Steps 2-5 until all territories allotted for survey in a given day are visited. Ground 
and sUAS may not survey territories allotted for other survey days to maintain separation of survey 
platforms. 


Step 7: Record daily “Survey End Time” on return to English Village or runway. 


“In-Use Nest” Protocol (to be used with both “Early Season” and “Late Season” surveys): 
Surveyors will determine status of newly or previously found In-Use Nest by watching for up to 
60 mins (but ended sooner if definitive status is reached sooner, or maximum 4-hour territory time 
expires). 


e For newly discovered “In-Use Nest”, 60-minute clock begins when nest first verified used 
(e.g., may exclude up to 30 mins of survey while still “unknown”’); for all others, time 
begins with arrival at view coordinate. 


e If two adults seen on the nest, watch until confirmation of incubation/brooding, eggs, 
nestlings, fledglings, or failure (proceed to those statuses below), or list as “adults on nest- 
unknown” if status unresolved after 60 mins or when time expires. 


e Incubation/brooding: if adult seen in incubating/brooding posture, record status and end 
nest survey. 


e Egg(s): if egg(s) seen, record number, presence of adult(s), other circumstances, and end 
nest survey. 


e Nestling(s): if nestling(s) seen after March 24, spend up to 60 mins attempting to count and 
age to nearest half-week. Surveys may be ended sooner without a count of nestlings when 
all-downy eaglet(s) (<4 weeks) seen. Note presence of adults, feeding events, etc. List as 
“unknown” age and/or count if unresolved after 60 mins or time expires (or note if not 
attempted due to pre-March 24 date). If age is determined to be 7 weeks or older (80% 
fledge age), proceed to “fledgling” protocol below. 
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e Fledgling(s): spend up to 60 mins attempting to determine total number of fledglings 
reaching at least 80% fledge age (7 or more weeks). Annual nest survey effort is complete 
when fledge age reached and confident fledgling count achieved (may require subsequent 
week visit). 


e Failure: nest confirmed as empty of adults, eggs, or nestlings, after previous visit confirmed 
In-Use (excluding “adults on nest-unknown”), or status of previously confirmed In-Use 
Nest cannot be verified after two consecutive weeks of 60 min survey. 


e Previously In-Use-unknown: status cannot be verified after 60 mins of watching, despite 
activity confirmed during a previous survey. Two consecutive nest surveys with this status 
results in assigning failure. 


5.5 SAMPLING PROTOCOL 


Laboratory samples were not collected during this demonstration. Field survey methods are 
addressed in section 5.4. 


5.6 SAMPLING RESULTS 


Laboratory samples were not assessed during this demonstration. Field survey results are 
addressed in section 6.0. 
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6.0 PERFORMANCE ASSESSMENT 


The demonstration team successfully completed two full seasons of eagle nest monitoring on DPG 
during 2019 and 2020 (Table 4). However, eagle nesting initiation and success rates were very low 
across the entire Utah West Desert region during both years (HawkWatch International 2019). 
Eagle nesting initiation is currently thought to be a function of available food supply, namely 
black-tailed jackrabbits for the West Desert population of golden eagles (Steenhof et al. 1997). 
Jackrabbit populations in Utah are currently at a low in their population cycle, resulting in a few 
years of depressed golden eagle nest initiation and success (HawkWatch International 2019). Eagle 
nesting success may begin to trend upwards as soon as next year (2021) as rabbit populations 
rebound (Bedrosian et al. 2017). It may be that better eagle nesting success during monitoring 
would allow for a larger dataset and more robust conclusions regarding the abilities of each 
monitoring platform. 


6.1 STATISTICAL ANALYSIS 


As a result of the low nesting initiation and success rates on DPG during both monitoring seasons, 
the demonstration team was not able to collect enough independent data points to support a robust 
mixed effect regression model to assess platform success as we had proposed. The large number 
of variables involved in our proposed model demand a large set of independent nest observations, 
and a large number of replications of those observations. We only observed two in-use nests in 
2019, and 3 in 2020 (although only one nest in 2020 survived to fledging age). Instead, the DPG 
Team developed a simplified version of the proposed model focusing on general nest observations 
and status assessment. Survey platform was treated as a fixed effect with the binary response 
variable being correct or incorrect weekly nest status classification within each territory based on 
the “true” weekly status. Territory, survey week, and period (early/late) were treated as random 
effects. Mixed effect modeling allowed us to assess whether the survey platform influenced the 
odds of correctly classifying weekly statuses, outside the random effects of time and territory. In 
addition, the DPG Team created a logistic regression model to assess the effects of survey platform 
on the amount of time required to determine nest status. Survey platform was treated as a fixed 
effect with the number of minutes required to determine each nest status as the response. We 
included territory, survey week, and nesting period as additional variables. We ran two separate 
runs of both models for each year, as the differences in sUAS platforms greatly affected the results 
from year to year. 


The initial proposal was to compare military UAS platforms to the sUAS and ground observer 
teams directly. However, it was determined in the first season of operation (2019) that flight 
schedules for the military UAS platforms were too sporadic to support regular nest monitoring. 
The military UAS were only able to monitor nests on an as-available basis, and other missions, 
weather, and maintenance concerns frequently bumped nest observations off the schedule. In 
addition, the COVID-19 crisis in 2020 resulted in the UAS observer being barred from conducting 
nest observations within the UAS control box due to health and safety concerns. Due to a lack of 
repeated observations on the same scale as the ground observer and sUAS, the military UAS 
platform was not included in the comparison models. 
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Table 4. Nest Monitoring Results of Both Survey Years, Broken Out by Year and 
Platform, and Including “True” Nest Statuses and Dates as Determined by Backdating and 
Remote Cameras Placed in In-use Nests 


Metric Year | “TRUE”| Ground | sUAS UAS | Comment 
# Territories 2019 11 11 11 6 
2020 11 10 11 NA 
# Occupied Territories 2019 11 10 11 1 
2020 6 6 6 NA 
# Nests Surveyed 2019 85 59 63 11 
2020 85 47 62 NA 
Average Weekly Nests 2019 NA 16 27 1 
Surveyed 
2020 NA 15 26 NA 
# In-Use Nests 2019 2 2 2 1 
2020 3 2 2 0 
# Fledglings 2019 2 2 2 1 
2020 2 2 0 NA 
% Success 2019 100% 100% 100% 100% 
2020 33% 50% 0% NA 
Fledglings/Adult Pair 2019 1 1 1 NA 
2020 0.33 0.33 0 NA 
Territory Occupancy 2019 NA 34 32 1 Total number of early season 
Sample Units territory surveys in which definitive 
territory occupancy (excluding In- 
Use Nests) documented 
2020 NA 33 31 NA 
In-Use Nest Sample 2019 NA 21 21 2 Total number of territory surveys in 
Units which definitive In-Use Nest 
documented 
2020 NA 13 6 NA 
% correct closure 2019 100% 81% 89% 25% |% of In-Use Nest survey dates 
correctly assigned closure 
2020 100% 93% 83% NA 
% incorrect closure 2019 0% 19% 11% 75% | % of nest survey dates incorrectly 
assigned closure (too early/too late) 
2020 0% 7% 17% NA 
Field Survey Man- 2019 NA 8:39 18:00 1:22 | Average weekly driving/flight and 
Hours survey hours (“Survey Time” x 
number of personnel) 
2020 NA 7:47 20:09 NA 
Territory Survey Man- 2019 NA 1:38 4:09 0:25 | Average time within 2-km geofence 
Hours x number of personnel 
2020 NA 1:47 3:54 NA 
Nest Survey Time 2019 NA 0:17 0:09 0:07 | Average weekly time needed to 
determine nest status 
2020 NA 0:20 0:11 NA 
New Nests Found 2019 NA 1 3 0 Number of previously unknown 
nests identified 
2020 NA 1 3 NA 
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6.2 TIME TO EARLY SEASON OCCUPANCY STATUS 


The ground observer and the sUAS took roughly the same amount of time to determine early 
season territory occupancy each year, even with the change in sUAS platform in 2020. The ground 
observer took a mean time of 111 minutes to determine occupancy in 2019 (SD = +-64) and 108 
minutes in 2020 (SD = +- 64). The sUAS observer took a mean time of 108 minutes in 2019 (SD 
= +- 54) and 91 minutes in 2020 (SD = +- 50). The UAS took vastly less time to make a territory 
determination in 2019, although this is mostly due to the limited available flight time the platform 
had to work within. The UAS platform was rarely able to make definitive determinations within 
the given time allowed and was removed from the comparison. Most of the time required by both 
the sUAS and ground observer was taken by drive time between nests in a territory (Figure 7). 


Time To Determine Early Season Status 
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Figure 7. Chart Showing the Mean Time in Minutes (+SD) each Platform Required to 
Determine the Early Season Occupancy of a Territory. 


6.3 NUMBER OF IN-USE NESTS IDENTIFIED 


Both the ground observer and the sUAS found the same number of in-use nests each nesting 
season, while the UAS was unable to locate one of the in-use nests in 2019 (the UAS was forced 
to cease operations before in-use nesting in 2020 due to COVID-19). An interesting point to be 
considered is that in 2020, there were actually 3 in-use nests. The ground observer located one in- 
use nest that the sUAS was unable to observe due to territorial aggression against the platform by 
the eagle pair, while the sUAS located an in-use nest that the ground observer was unable to survey 
due to lack of ground access in an Impact Area. This illustrates the fact that both survey methods 
have strengths and weaknesses that the other can supplement (Figure 8). 
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Figure 8. Chart Showing the Number of In-use Nests Each Platform Identified During 
Each Breeding Season. 


6.4 DATE OF IN-USE NEST IDENTIFICATION 


The sUAS platform was roughly equal with the ground observer at detecting in-use nests in 2019. 
The sUAS platform was able to survey almost every known nest in each assigned territory each 
week, making it more likely for the platform to detect an incubating eagle within a week of nesting 
initiation. The ground observer, on the other hand, was forced to rotate which nests were surveyed 
during each visit due to long hikes over rough terrain. The 2019 UAS was typically not able to 
identify nest activity due to the low resolution of its payload and the lack of precise nest metadata 
provided to its pilots and was not compared. In 2020 the sUAS was again roughly equal to the 
gound observer in nest detection time, despite the sUAS’s drop in payload resolution. This 
indicates that the advantage of visiting more nests outweighed the disadvantage of poor payload 
resolution. In 2020, the UAS platform was forced to cease surveys before nests became in-use, 
removing it from this comparison (Figure 9). 
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Figure 9. Chart Showing the Mean Number of Days (+SD) from Nest Initiation to Activity 
Detection by Each Platform. 


6.5 DATE OF NEST FAILURE OR FLEDGE IDENTIFICATION 


In 2019, the ground observer and sUAS platforms very nearly tied for speed at detecting nest 
failure or fledge. Both had sufficient quality optics and visitation rates to detect the lack of chicks 
in a nest almost immediately. The UAS was not compared due to lack of data. By contrast, the 
ground observer was much faster to detect nest failure or fledge in 2020 than the sUAS. The 
sUAS’s loss of payload resolution made it difficult for the platform to identify the presence or 
absence of small chicks in a nest. Again, the UAS was forced to cease surveys before nests became 


in-use, removing it from this comparison (Figure 10). 
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Figure 10. Chart Showing the Mean Number of Days (+SD) from Nest Failure or Fledge to 
Detection by Each Platform. 


Error bars represent standard deviation. 
6.6 TIME TO NEST STATUS UPDATE 


The following logistic regression models illustrate the differences in nest observation efficiency 
both between platforms and between years. In 2019, the sUAS displayed a clear efficiency 
advantage over the ground observer, requiring much less observation time to make a nest status 
determination, especially in the early nesting season. This is because of the sUAS’s ability to view 
directly into a nest bowl, eliminating the need to wait for a possible incubating eagle to stand or 
stretch in order to come into view from the ground. As the nesting season went on and chicks got 
larger, they were easier to see from below, decreasing the advantage of the sUAS. In 2020, the 
sUAS retained a small advantage over the ground observer, but the poor resolution of the payload 
meant that the platform had to observe each nest for much longer in order to determine if an object 
perceived to be in the nest was actually an eagle or if it was simply a pixilation artifact. In addition, 
the ground observer in 2020 took more time on average as the season went on, mostly due to the 
fact that one of the in-use nests was located in a high, partially obscured cliff cavity, preventing 
easy observation at any time. 
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Figure 11. Comparison of the Efficiency of a Ground Observer in Minutes (+ 95% 
Confidence Intervals) vs an sUAS at Determining Any Given Nest’s Status During 2019. 
The sUAS took vastly less time to determine status during early season, while the ground observer gained 
efficiency as unused nests were eliminated from the survey round. 
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Figure 12. Comparison of the Efficiency of a Ground Observer in Minutes (+ 95% 
Confidence Intervals) vs an sUAS at Determining Any Given Nest’s Status During 2020. 
The sUAS took less time to determine status on average. The sUAS’s efficiency advantage was much less 
pronounced than during 2019. 


6.7 NUMBER OF NEW NESTS FOUND 


In both 2019 and 2020, the sUAS outcompeted the ground observer and the UAS at finding 
previously undescribed nests. The sUAS had the advantage of covering a great deal of territory 
quickly, while also taking photos and video of terrain to be later analyzed for potential nests. The 
ground observer was unable to visit as much of each territory and had to focus on observing already 
known nests, preventing the observer from spending much time scanning for new nests. The UAS 
was unsuited to locating new nests due to the complications of approach angle and low payload 
resolution in 2019 and did not have enough survey time to draw a comparison in 2020 (Figure 13). 


29 


New Nests Detected 


3.5 


15 E 2019 


Ground sUAS UAS 


# of Nests 


m 2020 


0.5 


Platform 


Figure 13. Chart Showing the Number of New Nests Each Platform Located During Each 
Season. 


6.8 HYPOTHETICAL MISSION IMPACT IN DAYS 


The hypothetical mission impact of nest area closures and openings is a measure of the precision 
of each platform in detecting nest status changes. Ideally, an in-use nest will be detected, and an 
area closure enacted immediately (within one week or survey period) upon the start of incubation, 
and the area opened again immediately upon the failure or fledge of the nest. This minimizes 
potential mission impacts of unnecessary range closures as well as protecting the nest against 
potential disturbance (Figure 14). 


In 2019, both the ground observer and sUAS platform were slow to detect the activity of RN315, 
leaving it unprotected for several weeks, although the sUAS was one week quicker in detecting 
the nest’s activity. In addition, the ground observer designated RN315 as fledged prematurely by 
one week. This resulted in the sUAS being cumulatively more precise in designating range closures 
in 2019 than the ground observer. 


In 2020, the data is complicated by the fact that both the ground observer and the sUAS monitored 
nests that the other platform was not capable of reaching, leaving only one nest for direct 
comparison. The ground observer was slow to detect the activity of RN248A by several weeks but 
caught the fledge of the nest immediately. The rough terrain and lack of road access in RN248A’s 
territory required the ground observer to ration which nests received a survey visit each week, 
which resulted in the in-use nest going undetected for several weeks. The sUAS detected the 
activity of RN029 immediately but was slow to confirm the failure of the nest by several weeks. 
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The low resolution of the platform payload in 2020 made it much more difficult to identify the 
presence or absence of young chicks from the nest bowl than the presence or absence of an 
incubating adult. Both platforms were equally precise at determining the status of RN321. 


Hypothetical Mission Impact of Nest Protections 
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Figure 14. Chart Showing the Hypothetical Impact of Nest Area Closures/Openings by 
Platform and Year. 


The 0 line represents no deviation from actual nest use, i.e., the desired perfect protection. 
6.9 QUALITY OF PHOTO/VIDEO DOCUMENTATION 


The DPG Team conducted a qualitative evaluation of how well each platform was able to 
document nest status in both 2019 and 2020. In 2019, the sUAS platform delivered high resolution 
video and photo documentation of all nests viewed, as well as footage of the surrounding cliffs for 
the team to search for previously unknown nests. The photo/video documentation allowed the data 
managers to confirm nest status before putting range closures in place. By contrast, the UAS 
platform in 2019 was equipped with a low-resolution payload that hampered the observer”s ability 
to identify nests with confidence. The UAS”s thermal imaging capability did allow the observer to 
spot chicks in in-use nests, however. In 2020, the imaging capabilities of the sUAS and UAS 
platforms were reversed. The replacement sUAS platform had a downgraded payload resolution 
due to downlink limitations. In addition, it did not have onboard data recording capabilities; the 
observation team had to record footage from the ground station, and as a result encountered 
significant issues with poor reception and signal corruption. These limitations resulted in the sUAS 
team requiring more time on site to determine nest status, and in many cases prevented the team 
from calling a certain nest status at all. On the other hand, the UAS platform received a significant 
upgrade to its payload systems, allowing for HD image recording. Unfortunately, the UAS team was 
forced to cease survey operations after only 2 weeks of monitoring due to the COVID-19 crisis, and 
the observation team was unable to obtain enough data to present a significant image comparison. 
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The photos below show the images taken from the two sUAS and the one UAS (Figures 15-18). 


Figure 15. A Photo of RN315 as Taken by the sUAS Platform in 2019. 


(DJI Matrice 600 Pro with Zenmuse Z30 payload). The photo shows an adult golden eagle sheltering a 
chick less than a week old. 


Figure 16. A Series of two photos taken by a ground observer at two different zoom levels 
of RN046. 


The first photo shows the distance the observer must view the nest from, as well as the height of the nest 
on the cliff face. The second photo shows how the ground observer must view the nest from below, even at 
high zoom. 
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Figure 17. A Video Still Taken from the Footage of the UAS Observer Using Thermal 
Imaging in 2019, Showing RN315 with a Nearly 9-week-old Chick. 


The chick is shown as a pale (warm) shape in the middle of a cool cliff face. This image shows the 
challenges the UAS observer faced with low resolution imagery, and how thermal imaging was able to 
partially compensate in the late nesting season. 


Figure 18. A Video Still Taken from the Footage of the sUAS Observer in 2020 (E900 with 
Sony FCB-EV7520A), Viewing Nest RN248. 


The photo shows the low resolution and video artifacts that inhibited the sUAS’s ability to make accurate nest 
activity identifications in 2020. 
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6.10 ADDITIONAL METRICS FOR SUCCESS 


The results of the mixed-effect logistic regression models indicate that the ground observer and 
sUAS platform had essentially equally (high) accuracy in determining the status of any given nest 
in 2019. Both platforms were more than 85% likely to determine nest status accurately and 
confidently in the early season, growing to more than 95% accurate in the late season. This can be 
attributed to the experience and skill of the ground observer, as well as the ability of the sUAS 
platform to take high quality documentation photos and video for review. In 2020, the sUAS 
platform suffered greatly from the drop in payload resolution. It began strongly and was still 
confidently able to determine when nests were not in-use. However, it struggled to accurately 
identify when nests were in-use and to determine the difference between an incubating nest and a 
nest with chicks. The ground observer, on the other hand, displayed almost the same results as in 
2019 (Figures 19 & 20). 
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Figure 19. Comparison of the Accuracy of a Ground Observer (+ 95% Confidence 
Intervals) vs an SUAS at Determining Any Given Nest’s Status During 2019 


Both platforms gained accuracy as the season went on, roughly equal in accuracy. 
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Figure 20. Comparison of the Accuracy of a Ground Observer (+ 95% Confidence 
Intervals) vs an sUAS at Determining Any Given Nest’s Status During 2020. 


The ground observer gained accuracy during the season, mirroring 2019, while the sUAS lost accuracy. 
This represents the challenges of identifying an in-use nest’s status with a low-resolution payload. 


When determining which platform is most efficient and cost-effective, it is important to take total 
territory coverage into account, especially in the early nesting season when looking to identify in- 
use nests. The sUAS platform was able to survey approximately twice as many nests each week 
than the ground observer in both 2019 and 2020. This grants the sUAS more confidence in 
accurately calling the occupancy status of early season territories, as well as giving it an advantage 
in locating in-use nests promptly. This played out in 2019 as the sUAS located one in-use nest a 
full week before the ground observer. The ground observer had to rotate which nests to observe 
each week due to time pressure and the lengthy hikes needed to cover some territories. By being 
unable to cover all known nests each week, the ground observer was slower to identify an in-use 
nest, leaving it potentially vulnerable to disturbance (Figure 21). 
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Figure 21. Chart Showing the Mean Number of Nests (+SD) Surveyed Each Week During 
the Early Season. 


There is high variation from week to week depending on which territories were assigned for survey, but 
the sUAS consistently outperformed the ground observer over the same territory assignments. 
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7.0 COST ASSESSMENT 


The cost assessment included actual costs of both the sUAS and the on-the-ground observer. 
However, the actual costs are skewed based on a few circumstances that were out of the ordinary 
for this project. Just as this demonstration was beginning, the DoD memo was released banning 
COTS sUAS. The cost of the modified COTS sUAS included non-recurring engineering costs that 
would not have been included in a COTS sUAS. The changes in policy also required assistance 
from a Government organization to prepare and obtain a waiver for the project and provide direct 
oversight of the flights. 


7.1 COST MODEL 


The tables below break out the costs for each of the monitoring methods for comparison. Due to 
the significant differences between the two sUAS these were also broken out for comparison. Table 
5 provides the costs for year one using the DJI Matrice and Table 6 provides that costs for year 
two using the E900. Both tables show the cost of Government support for waiver preparation and 
sponsor/oversight of flights. This cost should not necessarily be considered for future projects. 


Table 5. One Year Cost Model for an sUAS. 


Cost Element Data Tracked During the Demonstration Estimated Costs 


Purchase of sUAS 
Platform : . $18,139 
*Modified commercial sUAS (DJI Matrice) 


Payload Purchase of camera NA- Integrated into 


platform 
Batteries/Chargers Purchase of extra batteries/chargers $1,528 
Binoculars/Scope Not Applicable for sUAS NA 
Transportation Mileage to-from observation locations $3,000 
Maintenance/Repair Purchase of repair kit and replacement parts $256 
Monitoring Labor Team of three $55,346 
Waiver Development Development of waiver request memo and $7,000 
(Contractor) supporting documentation 


Govt. waiver prep and 
sponsor/training/oversight of | Cost of Govt. support for waiver 
flights 


$79,557 


Total $164,826 
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Table 6. One Year Cost Model for an sUAS 


Cost Element Data Tracked During the Demonstration Estimated Costs 
Purchase of sUAS 
Platform $5,500 


*Modified commercial sUAS (E900) 


Govt. waiver prep and sponsor/ 


training/oversight of flights Cost of Govt. support for waiver $61,249 


Batteries/Chargers $1,651 
Binoculars/Scope NA 
Transportation $3,000 
Maintenance/Repair $893 
Monitoring Labor $48,840 
Waiver Development Development of waiver request memo and 


Total $134,028 


Table 7 below shows the costs for the military UAS. Because the military UAS was used on an as 
available basis the costs of the platform/payload were not applicable. Only the cost associated with 
the biologist that sat with the UAS pilots was calculated. The cost of a military UAS not be cost- 
effective for natural resource monitoring alone. However, if an installation has a UAS program and 
the platform can be made available as part of the pilot training program, it is a useful tool to have. 


Table 7. One Year Cost Model for an UAS 


Cost Element | Data Tracked During the Demonstration | Estimated Costs 
Platform Not Available NA 
Batteries/Chargers NA 
Binoculars/Scope NA 
Maintenance/Repair NA 

Monitoring Labor $10,910 

Waiver Development (Contractor) NA 
Sener al NA 

Total $10,910 


Table 8 below provides a detail on the costs for on the ground monitoring. This was performed by 
one biologist for the two-year study. 


Table 8. One Year? Cost Model for On-the-Ground Monitor 


Cost Element Data Tracked During the Demonstration | Estimated Costs 
Platform Not Tracked NA 
Payload Not Tracked NA 


Z 
> 


Batteries/Chargers Not Tracked 
Binoculars/Scope Approximate Cost $2,000 
Transportation To and from observation locations $3,130 
Maintenance/Repair Not Tracked 

Monitoring Labor One Biologist $14,715 


Z 
> 


Waiver Development (Contractor) | Not Tracked 


Govt. waiver prep and sponsor/ 
training/oversight of flights 


Total $19,845 


Not Tracked 


Z 
> 


Z 
> 


Table 9 below shows the cost of using helicopters for monitoring. These costs are based on 
historical costs since helicopters were not used during the two-year study. 


Table 9. One Year? Cost Model for a Helicopter 


Cost Element | Data Tracked During the Demonstration Estimated Costs 
Platform oe cost of $1,500 for 36 days (4 hours/ $216,000 
Batteries/Chargers NA 
Binoculars/Scope $2,000 
Maintenance/Repair NA 

Monitoring Labor $8,640 

Waiver Development (Contractor) NA 
A NA 

Total $226,640 


The two tables below provide a side-by-side comparison of the two sUAS, UAS, on-the-ground 
monitor and helicopter. Table 10 includes the cost of Government support to prepare, obtain a 
waiver for the sUAS and monitor flights, while Table 11 removes those costs. Due to the unique 
circumstances faced during this demonstration, it is assumed that the additional cost for 
Government support for waiver and oversight would not be necessary for future projects. 


Table 10. Cost Comparison of Methods with Government Support for Waiver 


sUAS-DJI 
Cost Elements Matrice sUAS- E900 On-The-Ground | Helicopter 


Platform 
Payload NA $5,895 
BateriesiCharger 
Binoculars, scope 5 a A 


Transportation (to-from 

observation locations ) $ 3,000 | $ ll ===] 3, = 

Maintenance/Repair $ 256 | $ 893 | NA | 
$ 


Monitoring Labor $ 55,346 48,840 “a 910 mM 715 — 640 
Waiver Development (contractor) | $ 7,000 | $ 7,000 Fe ean apagar pagas 


*Govt. waiver prep and 
sponsor/training/oversight of flights | $ 79,557 | $ 61,249 | NA a NA 
Total $ 164,826 | $ 134,028 | $ 10,910 19,845 | $ 226,640 


* Cost specific to this project as a result of DoD Memo banning COTS sUAS and m NDAA prohibiting the 
procurement or operation of Chinese made sUAS. 


Z 


Table 11. Cost Comparison of Methods without Government Support for Waiver 


sUAS-DJI 
Cost Elements Matrice =~ EE a Ground CEET 


na [5 216,000, 
CE CA E CU CC E 
bona =p ai e a 


Transporation (to-from 
observation locations ) 3 mn $ 3 ma 3,130 


Maintenance/Repair $ 
Monitoring Labor $ 55, ae : 48, 10 T 910 = 14,715 | $ 8,640 


Waiver Development (contractor) | $ 

*Govt. waiver prep and 

sponsor/training/oversight of flights - NA 

Total 85,269 | $ 72,779 À Im 910 19,845 | $ 226,640 


* Cost specific to this project as a result of DoD Memo banning COTS sUAS and TT NDAA prohibiting the 
procurement or operation of Chinese made sUAS. 
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Cost Elements: 


sUAS Platform: The sUAS platform is a primary cost. The platform must be able to meet, in this 
case, DoD Army cybersecurity and procurement requirements while also being able to 
accommodate the necessary payload. Performance, durability, and flight time are all important 
factors when selecting a platform. 


Costs for the sUAS DJI Matrice 600: This is based on actual costs of the two DJI Matrice 
acquired from the Government. The DJI Matrice 600 were COTs that had been modified to use a 
U.S. developed software for control. This modification increased the cost of the sUAS. 


Costs for the sUAS E900: This was an estimated cost provided from the Government. The E900 
used in the second year were loaned to the program. The E900s used in this project could be 
considered developmental. The payload was integrated into the system and was not part of the 
original system. 


sUAS Payload: The sUAS payload includes the camera and/or sensors. The payload can be a 
significant cost depending upon the requirement and whether or not it has been integrated into the 
platform. The payload weight will also need to be considered when selecting an sUAS platform. 


Batteries/Chargers: Batteries are critical for the operation of the sUAS and are a significant cost. 
Back up batteries are necessary to ensure flights can continue without downtime. Charging 
infrastructure for the batteries was also a consideration. Because the sUAS are used in remote 
locations, it is necessary to have portable charging for the batteries. 


Binoculars/Scopes: High quality binoculars and scopes are necessary for on-the-ground and 
helicopter monitors. 


Transportation: Both sUAS and on-the ground monitoring requires transportation from a starting 
location to the monitoring or launch location. 


Maintenance/Repair: Although little maintenance or repair was performed during this 
demonstration, it is important to consider when implementing a program. Common 
maintenance/repairs include battery charging, loose bolts/fittings, broken rotors, landing gear, etc. 
Each platform was only operated for a few months in each year. Industry data suggests the life 
cycle of the typical sUAS to be approximately 200 flights. 


Monitoring Personnel: Personnel for the sUAS consisting of a pilot, an operator, a biologist, and 
observer. The observer was a person associated with the waiver who ensured that all requirements 
were being met. However, based on experience, the four people could be reduced to two 
(pilot/biologist). 


Waiver Development (Contractor): The Exemption to Policy (ETP) waiver process includes the 
technical documentation of the sUAS to obtain an Airworthiness Release (AWR). Once the AWR is 
in place, the development and approval of standard operating procedures, System Safety Management 
Plan, risk acceptance memos, and the formal ETP memo signed by a senior official may be prepared. 
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The preparation of this material, the submission through a SIPR account, discussions/clarifications 
from the Army CIO, staffing of memos for signature, all take time and have an associated cost. 
The process to obtain an ETP changed three times from 2018 to 2020. 


Access to Installation and Logistics: This cost is associated with obtaining base access and 
coordination with range control and flight control. 


Government Waiver Preparation and Sponsorship/Training/Oversight of Flights: This cost 
may be eliminated or significantly reduced in the future. During this demonstration, it became 
necessary to engage with another Government organization to sponsor the waiver and to support 
its development. In addition, this organization was able to provide a modified sUAS that was 
capable of meeting DoD cyber requirements. Because they were the sponsor of the waiver, they 
were on-site to provide training on the sUAS and weekly flights. 


7.2 COST DRIVERS 


The most significant cost driver is DoD Army sUAS policy. The platform/payload must meet 
Department of Army cyber security requirements. Each platform must undergo an evaluation and 
provide an Exemption to Policy (Waiver) to operate in restricted airspace. The cost associated with 
obtaining an ETP includes labor to develop the ETP application package (Standard Operating 
Procedures, System Safety Management Plan, and Memos), upload through SIPR, and track 
through the system. The waivers are only for six months. In addition, each sUAS must have an 
Airworthiness Release. 


The prohibition on foreign made sUAS impacts costs because the U.S. market is still developing 
and has not yet produced a cost competitive commercial sUAS to compete with foreign made 
sUAS. As the U.S. market matures and purchases increase, this will begin to bring the costs down. 


7.3 COST ANALYSIS AND COMPARISON 


Costs are based on golden eagle monitoring on a MRTBF with vast distances between nests and 
rugged terrain. Breeding season begins in February and goes through June, allowing for a range of 
temperatures and weather conditions. 


The cost comparison assumes that both the sUAS and on-the-ground monitoring start and return 
from the same location. The time to survey nests and return to the starting location was recorded. 


Industry data on life expectancy of an sUAS is approximately 200 flights. When first envisioning 
this project, it was thought that the sUAS would be most cost-effective and could replace the 
current approach of on-the-ground monitoring. The results from the demonstration show that the 
use of an SUAS is a good tool to have available but may not be able to cost-effectively replace on- 
the-ground monitoring. For instances where the nests are too high or distances too long or rugged, 
sUAS are a good tool. As the cost of the sUAS system is reduced and the policy surrounding flying 
the sUAS in DoD airspace is streamlined or eliminated, it may become cost-effective compared to 
on-the-ground monitoring. The sUAS is more expensive than on-the-ground monitoring, however, 
the sUAS provides benefits over that of the on-the-ground monitor such as photos which can be 
used to study and accurately age the chicks. The sUAS can also cover more territory than an on- 
the-ground monitor and access nests that are too high or at a difficult angle to see from the ground. 


42 


In addition, some of the costs associated with this demonstration can be reduced or eliminated. For 
instance, a three-person team was used to monitor eagles (the pilot in charge, operator, and 
observer). In the future, this team could be reduced to a two-person team. In addition, the 
Government sponsor of the waiver observed weekly flights which required labor and travel. Due 
to COVID-19 in the 2020 season, the Government sponsor was able to designate a local 
Government Point of Contact (POC) to provide oversight which reduced costs. The use of sUAS 
compared to a helicopter is cost competitive. 
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8.0 IMPLEMENTATION ISSUES 
8.1 SUAS IMPLEMENTATION 


A major implementation issue with the use of sUAS in DoD airspace involves obtaining an 
Airworthiness Release (AWR) and Exemption to Policy (ETP). This policy seeks to prevent the 
use of foreign made sUAS based on cybersecurity vulnerabilities. Both approvals take time to 
develop, review, and obtain approval. Under the 2019 DoD Army policy, a modified DJI Matrice 
600 was used to monitor golden eagles after obtaining an ETP. However, because of the FY 2020 
NDAA, Section 848 Prohibition on Operation or Procurement of Foreign Made Unmanned 
Aircraft Systems, the procurement or use of Chinese made sUAS was prohibited. This led to the 
use of a different sUAS platform. Each change of platform requires a new AWR and ETP. The 
ETP can only be granted for six months at a time and the time to obtain an approval may take 45- 
60 days minimum. 


To obtain an Airworthiness Release, a Standard Operating Procedure, System Safety Management 
Plan, Material Risk Acceptance Memo, and Operational Risk Acceptance Memo, must be 
developed and submitted to the Aviation Engineering Directorate. Once the AWR is provided, it 
is sent along with the other documents with a memo, signed by a Senior Level memo requesting 
an ETP. The AWR is valid for three years. 


The sUAS platform used for observation and data collection has real potential for becoming a 
quick, cost-effective way to monitor the progress of nesting birds in hard to reach places. For this 
platform to work efficiently and in a manner that collects the best data, some points described 
below have been found over the past two years to help improve the use of this platform. 


Good metadata and a member of the team that has experience with eagle behavior and good 
knowledge of the territory is key to obtaining useful data. A trained eye for these sUAS platforms 
was able to compensate for finding nests with sometimes less than ideal video streaming quality 
or lighting situations. A profile containing pictures from multiple angles and distances and GPS 
coordinates of known nest in the territory was critical to confirming nest locations and to reduce 
the time that the sUAS team spent locating nests during a flight. 


For the legal operation of the sUAS to fly commercially, Federal Aviation Administration (FAA) 
regulations require one member of the team to be Part 107 rated as a commercial remote pilot. 
While the Part 107 remote pilot does not need to be on the controls during flight, he/she must be 
present and is responsible for lawful compliance of the operation; or in other terms, is serving as 
Pilot in Command (PIC). To comply with Part 107 while also maintaining the ability to collect 
high quality data, the sUAS must have some certain technological capabilities. For instance, per 
Part 107 the sUAS must remain in Visual Line-Of-Sight (VLOS) of the PIC, the person 
manipulating the controls, and a visual observer without the use of any vision aiding device other 
than corrective lenses. The sUAS must also remain an adequate distance from a nesting site to not 
“take” the eagles as per BGEPA requirements. Therefore, to comply with both regulations, a long- 
range transceiver for video transmission, as well as a high optical zoom HD camera, is necessary 
to record proper data. 
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Both platforms have stabilization systems utilizing software that controls a three-axis gimballed 
camera mount. An HD camera with a gimbal mount was key to obtaining a clear picture for data 
collection from long distances and in variable wind conditions. The camera for both sUAS 
platforms had a 30x zoom which allowed the sUAS to maintain legal distance from nesting sites 
to avoid disturbing nesting eagles. Long distance video transmission capabilities, in this case 
greater than one mile, also aided in keeping distance from nesting birds. Both cameras on the 
platforms had auto focus, though the DJI also provided an intuitive touchscreen focus selector to 
allow operators to quickly refocus on any part of the camera view angle. In addition, DJI had auto 
and manual exposure, which was very effective in compensating for harsh shadows and high 
contrast lighting on cliff faces. Both systems offered the ability to record both photo and video 
data. However, the DJI provided direct on-board data recording while the E900 only allowed for 
data to be recorded from the receiver, which led to corruption and degradation of the photos and 
video due to radio interference. Direct on-board data recording capabilities are vastly preferable 
as a means of retaining high-quality photos and video. On the other hand, because of the E900’s 
modular components and features, it is able to offer the ability to slave two different controllers to 
the sUAS which proved beneficial. The ornithologist was able to control the camera gimbal and 
zoom to find nests with one controller, while the operator was able to fly the sUAS into viewing 
position using a separate controller. 


As mentioned in section 4.2, Dugway has a wide range of temperatures and weather conditions 
during the observation period. It was found during use of the sUAS that cold temperatures 
adversely affect battery performance and consequentially the amount of time the drone could spend 
per flight investigating a territory. This led to increased time in a territory as time would be needed 
for a battery swap and a second flight to obtain the necessary data. Other weather considerations 
included windy conditions at the surface as well as at observation altitude. These conditions 
required more finesse from the pilot, as well as requiring the sUAS to expend more battery power 
as the gimbal and stabilization systems on the platform worked harder to compensate for the windy 
conditions and maintain a fixed three-dimensional GPS location. The extra battery power required 
to hold location ended up limiting the loiter time and range of the sUAS. Comparing the two 
platforms that were used, it was found that the E900 had better battery power duration and a more 
compact battery system. The downside of this more compact system was a top-heavy design, 
which made the E900 more unstable and prone to tipping while landing in a crosswind. 


8.2 UAS IMPLEMENTATION 


The UAS observation method was the most uncertain upon the start of the 2019 nesting season 
and it proved to be unreliable as a method of weekly nest observations due to scheduling and 
payload restrictions. The realities of flight scheduling with RIAC students resulted in flight days 
that were sporadic. RIAC allowed the use of their students and platforms on an as-available basis, 
and other missions, weather concerns, or maintenance frequently bumped nest observations off the 
schedule. In addition, the sections of airspace the students were scheduled for were often not over 
the desired areas of range containing eagle nests to be surveyed. It became clear to the DPG Team 
that rather than using the UAS as a stand-alone observation method, it would be more useful as a 
precision tool for obtaining rapid nest observations in areas with no ground access or with critical 
range use planned within the next two weeks. 
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The UAS platform was unique in that it was able to view nests with no ground access within 2 
miles. It also possessed a highly useful thermal camera capability, which could identify a chick 
within a nest in seconds (given an appropriate time of day, before the cliff face has heated in the 
sun). However, the payloads carried by the UAS fleet in 2019 proved to be inadequate to observe 
most eagle nests due to low resolution imagery. The HD payload upgrade in 2020 improved the 
nest-monitoring potential of the platform immensely. The DPG Team does not recommend the use 
of UAS platforms for nest surveys unless equipped with HD payload capabilities. 


If the nest location was well known, the UAS was by far the fastest and most efficient observation 
method due to lack of drive and hiking time between observation points. However, the nature of 
the platform’s flight and approach patterns made detailed nest metadata critical to the success of 
the survey. UAS platforms must determine an approach angle and loiter point for a territory 
observation from several kilometers away, requiring the observation team to have precise 
knowledge of the nest’s location reference data, elevation, aspect, bearing from the ideal 
viewpoint, and topographical inclusion before the nest can be found. The UAS observation team 
found that landscape level photos combined with detailed descriptions of a nest’s location were 
sufficient to allow for a successful survey. Without this metadata, the UAS platform was nearly 
incapable of locating a nest in the time allowed. 


The DPG Team found that the most important factor influencing the success of the UAS team was 
the integration of the biological observer into the military framework of the RIAC students. The 
civilian observer had to learn the correct protocols and language, as well as establish trust and 
communication pathways with both the students and the instructors. Once the observer was able 
to work within the RIAC team, the frequency and success of survey flights increased dramatically. 
The RIAC students and instructors also found the nest surveys to be useful training experiences. 
Eagle nests are small, cryptic targets for a UAS operator, which presented the students with 
invaluable practical experience. The nest metadata was also presented in the style of a mission 
packet, allowing the students to practice working with external intelligence to locate a target. The 
DPG Team highly recommends nest monitoring as a dual-purpose use of UAS platforms, for both 
natural resource monitoring and student pilot training exercises. 
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APPENDIX B FY 2019 NATIONAL DEFENSE AUTHORIZATION ACT 


(NDAA) 
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1 (C) made available to appropriate govern- 
2 ment departments or agencies. 


OF FOREIGN-MADE UNMANNED AIRCRAFT — 
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(a) PROHIBITION ON AGENCY OPERATION OR PRO- 
7 CUREMENT.—The Secretary of Defense may not operate 


8 or enter into or renew a contract for the procurement of— 


9 (1) a covered unmanned aircraft system that— 
10 (A) is manufactured in a covered foreign 
11 | country or by an entity domiciled in a covered 
12 foreign country; 

13 (B) uses flight controllers, radios, data 
14 transmission devices, cameras, or gimbals man- 
15 ufactured in a covered foreign country or by an 

16 entity domiciled in a covered foreign country; 

17 (C) uses a ground control system or oper- 
18 ating software developed in a covered foreign 
19 country or by an entity domiciled in a covered 
20 foreign country; or 
21 (D) uses network connectivity or data aot 
22 age located in or administered by an entity 
23 domiciled in a covered foreign country; or 
24 (2) a system manufactured in a covered foreign 
25 country or by an entity domiciled in a covered for- 
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eign country for the detection .or identification of 


— 


covered unmanned aircraft systems. 

(b) EXEMPTION.—The Secretary of Defense is ex- 
empt from the restriction under subsection. (a) if the oper- 
ation or procurement is for the purposes of— 

(1) Counter-UAS surrogate testing and train- 
ing; or 


(2) intelligence, electronic warfare, and infor- 
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mation warfare operations, testing, analysis, and 


= 
© 


training. 


pa 
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(c) WAIVER.—The Secretary of Defense may waive 


= 
tb 


the restriction under subsection (a) on a case by case basis 


— 
uy 


by certifying in writing to the congressional defense com- 


P 


mittees that the operation or procurement is required in 


15 the national interest of the United States. 


16 (d) DEFINITIONS.—In this section: 
17 (1) COVERED FOREIGN COUNTRY.—The term 
18 “eovered foreign country” means the People’s Re- 
19 public of China. 
20 (2) COVERED UNMANNED AIRCRAFT SYSTEM.— 
21 The term “covered unmanned aircraft system” 
22 means an unmanned aircraft system and any related 
23 services and equipment. 
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